Mirrors/rapidyaml
0
0
Fork 0
mirror of https://github.com/biojppm/rapidyaml.git synced 2026-01-18 13:31:19 +01:00
Code Issues Packages Projects Releases Wiki Activity

minor fixes

Browse Source
This commit is contained in:
Joao Paulo Magalhaes
2024-06-02 13:32:40 +01:00
parent 4a9ebe5386
commit aad6ef3f71
13 changed files with 119 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files

94
README.md
View File

@@ -432,37 +432,6 @@ CHECK(loc.col == 4u);
## Using ryml in your project
### Package managers
ryml is available in most package managers (thanks to all the
contributors!) and linux distributions. But please be aware: those
packages are maintained downstream of this repository, so if you have
issues with the package, file a report with the respective maintainer.
Here's a quick roundup (not maintained):
* Package managers:
* [conan](https://conan.io/center/recipes/rapidyaml)
* [vcpkg](https://vcpkg.io/en/packages.html): `vcpkg install ryml`
* [PyPI](https://pypi.org/project/rapidyaml/)
* Linux distributions:
* Arch Linux/Manjaro:
* [rapidyaml (aarch64)](https://archlinuxarm.org/packages/aarch64/rapidyaml)
* [rapidyaml-git (AUR)](https://aur.archlinux.org/packages/rapidyaml-git/)
* [python-rapidyaml-git (AUR)](https://aur.archlinux.org/packages/python-rapidyaml-git/)
* [Fedora Linux](https://getfedora.org/)/[EPEL](https://docs.fedoraproject.org/en-US/epel/):
* `dnf install rapidyaml-devel`
* `dnf install python3-rapidyaml`
* [Gentoo](https://packages.gentoo.org/packages/dev-cpp/rapidyaml)
* [OpenSuse](https://build.openbuildservice.org/package/show/Emulators/rapidyaml)
* [Slackbuilds](https://slackbuilds.org/repository/15.0/libraries/rapidyaml/)
* [AltLinux](https://packages.altlinux.org/en/sisyphus/srpms/rapidyaml/3006055151670528141)
Although package managers are very useful for quickly getting up to
speed, the advised way is still to bring ryml as a submodule of your
project, building both together. This makes it easy to track any
upstream changes in ryml. Also, ryml is small and quick to build, so
there's not much of a cost for building it with your project.
### Single header file
ryml is provided chiefly as a cmake library project, but it can also
be used as a single header file, and there is a [tool to
@@ -531,6 +500,38 @@ If you omit `--recursive`, after cloning you
will have to do `git submodule update --init --recursive`
to ensure ryml's submodules are checked out.
### Package managers
ryml is available in most package managers (thanks to all the
contributors!) and linux distributions. But please be aware: those
packages are maintained downstream of this repository, so if you have
issues with the package, file a report with the respective maintainer.
Here's a quick roundup (not maintained):
* Package managers:
* [conan](https://conan.io/center/recipes/rapidyaml)
* [vcpkg](https://vcpkg.io/en/packages.html): `vcpkg install ryml`
* [PyPI](https://pypi.org/project/rapidyaml/)
* Linux distributions:
* Arch Linux/Manjaro:
* [rapidyaml (aarch64)](https://archlinuxarm.org/packages/aarch64/rapidyaml)
* [rapidyaml-git (AUR)](https://aur.archlinux.org/packages/rapidyaml-git/)
* [python-rapidyaml-git (AUR)](https://aur.archlinux.org/packages/python-rapidyaml-git/)
* [Fedora Linux](https://getfedora.org/)/[EPEL](https://docs.fedoraproject.org/en-US/epel/):
* `dnf install rapidyaml-devel`
* `dnf install python3-rapidyaml`
* [Gentoo](https://packages.gentoo.org/packages/dev-cpp/rapidyaml)
* [OpenSuse](https://build.openbuildservice.org/package/show/Emulators/rapidyaml)
* [Slackbuilds](https://slackbuilds.org/repository/15.0/libraries/rapidyaml/)
* [AltLinux](https://packages.altlinux.org/en/sisyphus/srpms/rapidyaml/3006055151670528141)
Although package managers are very useful for quickly getting up to
speed, the advised way is still to bring ryml as a submodule of your
project, building both together. This makes it easy to track any
upstream changes in ryml. Also, ryml is small and quick to build, so
there's not much of a cost for building it with your project.
### Quickstart samples
These samples show different ways of getting ryml into your application. All the
@@ -555,6 +556,7 @@ more about each sample:
| [`fetch_content`](./samples/fetch_content) | **yes** | [`CMakeLists.txt`](./samples/fetch_content/CMakeLists.txt) | [`run.sh`](./samples/fetch_content/run.sh) |
| [`find_package`](./samples/find_package) | **no**<br>needs prior install or package | [`CMakeLists.txt`](./samples/find_package/CMakeLists.txt) | [`run.sh`](./samples/find_package/run.sh) |
### CMake build settings for ryml
The following cmake variables can be used to control the build behavior of
ryml:
@@ -726,20 +728,16 @@ See also [the roadmap](./ROADMAP.md) for a list of future work.
### Known limitations
ryml deliberately makes no effort to follow the standard in the
ryml deliberately makes no effort to follow the YAML standard in the
following situations:
* ryml's tree does NOT accept containers are as mapping keys: keys
must be scalars. HOWEVER, this is a limitation only of the tree. The
event-based parser engine DOES parse container keys. The parser
engine is the result of a recent refactor and its usage is meant to
be used by other programming languages to create their native
data-structures. This engine is fully tested and fully conformant
(other than the general error permissiveness noted below). But
because it is recent, it is still undocumented, and it requires some
API cleanup before being ready for isolated use. Please get in touch
if you are interested in integrating the event-based parser engine
without the standalone `ryml::parse_*()`
* ryml's tree does NOT accept containers as map keys: keys stored in
the tree must always be scalars. HOWEVER, this is a limitation only
of the final tree. The event-based parse engine DOES parse container
keys, as it is is meant to be used by other programming languages to
create their native data-structures, and it is fully tested and
fully conformant (other than the general error permissiveness noted
below).
* Tab characters after `:` and `-` are not accepted tokens, unless
ryml is compiled with the macro `RYML_WITH_TAB_TOKENS`. This
requirement exists because checking for tabs introduces branching
@@ -774,11 +772,11 @@ following situations:
If you do run into trouble and would like to investigate conformance
of your YAML code, **beware** of existing online YAML linters, many of
which are not fully conformant. Instead, try using
[https://play.yaml.io](https://play.yaml.io), an amazing tool which
lets you dynamically input your YAML and continuously see the results
from all the existing parsers (kudos to @ingydotnet and the people
from the YAML test suite). And of course, if you detect anything wrong
with ryml, please [open an
[https://play.yaml.io](https://play.yaml.io), an amazingly useful tool
which lets you dynamically input your YAML and continuously see the
results from all the existing parsers (kudos to @ingydotnet and the
people from the YAML test suite). And of course, if you detect
anything wrong with ryml, please [open an
issue](https://github.com/biojppm/rapidyaml/issues) so that we can
improve.

2
ROADMAP.md
View File

@@ -1,2 +1,2 @@
Moved! See ryml's [Kanban board on github](https://github.com/biojppm/rapidyaml/projects/1).
Moved! See rapidyaml's [Kanban board on github](https://github.com/users/biojppm/projects/1/views/1).

4
doc/index.rst
View File

@@ -35,10 +35,10 @@ ryml is written in C++11, and compiles cleanly with:
* Visual Studio 2015 and later
* clang++ 3.9 and later
* g++ 4.8 and later
* clang++ 3.9 and later
* Intel Compiler
.. note::

2
doc/sphinx_quicklinks.rst
View File

@@ -13,7 +13,7 @@ Quick links
* `Pull Requests <https://github.com/biojppm/rapidyaml/pull>`_
* `Kanban board <https://github.com/biojppm/rapidyaml/projects/1>`_
* `Kanban board <https://github.com/users/biojppm/projects/1/views/1>`_
* Latest release: `0.6.0 <https://github.com/biojppm/rapidyaml/releases/tag/v0.6.0>`_

20
doc/sphinx_yaml_standard.rst
View File

@@ -17,7 +17,7 @@ welcome.
linters**, many of which are not fully conformant; instead, try
using `https://play.yaml.io
<https://play.yaml.io/main/parser?input=IyBFZGl0IE1lIQoKJVlBTUwgMS4yCi0tLQpmb286IEhlbGxvLCBZQU1MIQpiYXI6IFsxMjMsIHRydWVdCmJhejoKLSBvbmUKLSB0d28KLSBudWxsCg==>`__,
an amazing tool which lets you dynamically input your YAML and
an amazingly useful tool which lets you dynamically input your YAML and
continuously see the results from all the existing parsers (kudos
to @ingydotnet and the people from the YAML test suite). And of
course, if you detect anything wrong with ryml, please `open an
@@ -31,17 +31,13 @@ Deliberate deviations
ryml deliberately makes no effort to follow the standard in the
following situations:
- ryml's tree does NOT accept containers are as mapping keys: keys
must be scalars. HOWEVER, this is a limitation only of the tree. The
event-based parser engine DOES parse container keys. The parser
engine is the result of a recent refactor and its usage is meant to
be used by other programming languages to create their native
data-structures. This engine is fully tested and fully conformant
(other than the general error permissiveness noted below). But
because it is recent, it is still undocumented, and it requires some
API cleanup before being ready for isolated use. Please get in touch
if you are interested in integrating the event-based parser engine
without the standalone `ryml::parse_*()`
- ryml's tree does NOT accept containers as map keys: keys stored in
the tree must always be scalars. HOWEVER, this is a limitation only
of the final tree. The event-based parse engine DOES parse container
keys, as it is is meant to be used by other programming languages to
create their native data-structures, and it is fully tested and
fully conformant (other than the general error permissiveness noted
below).
- Tab characters after ``:`` and ``-`` are not accepted tokens, unless
ryml is compiled with the macro ``RYML_WITH_TAB_TOKENS``. This
requirement exists because checking for tabs introduces branching

10
src/c4/yml/detail/stack.hpp
View File

@@ -34,11 +34,11 @@ public:
public:
T m_buf[size_t(N)];
T * m_stack;
id_type m_size;
id_type m_capacity;
Callbacks m_callbacks;
T m_buf[size_t(N)];
T *C4_RESTRICT m_stack;
id_type m_size;
id_type m_capacity;
Callbacks m_callbacks;
public:

2
src/c4/yml/event_handler_stack.hpp
View File

@@ -46,7 +46,7 @@ public:
state *C4_RESTRICT m_curr; ///< current stack level: top of the stack. cached here for easier access.
state *C4_RESTRICT m_parent; ///< parent of the current stack level.
pfn_relocate_arena m_relocate_arena; ///< callback when the arena gets relocated
void *C4_RESTRICT m_relocate_arena_data;
void * m_relocate_arena_data;
protected:

26
test/test_parse_engine_6_qmrk.cpp
View File

@@ -292,6 +292,32 @@ ENGINE_TEST(Qmrk3,
___(ps.end_stream());
}
ENGINE_TEST(Qmrk4_0,
("[?baz:,]",
"[{?baz: }]"),
"+STR\n"
"+DOC\n"
"+SEQ []\n"
"+MAP {}\n"
"=VAL :?baz\n"
"=VAL :\n"
"-MAP\n"
"-SEQ\n"
"-DOC\n"
"-STR\n")
{
___(ps.begin_stream());
___(ps.begin_doc());
___(ps.begin_seq_val_flow());
___(ps.begin_map_val_flow());
___(ps.set_key_scalar_plain("?baz"));
___(ps.set_val_scalar_plain({}));
___(ps.end_map());
___(ps.end_seq());
___(ps.end_doc());
___(ps.end_stream());
}
ENGINE_TEST(Qmrk4,
("[ ? an explicit key, ? foo,? bar,?baz:,?bat]",
"[{an explicit key: },{foo: },{bar: },{?baz: },?bat]"),

2
test/test_suite/test_suite_event_handler.cpp
View File

@@ -72,7 +72,7 @@ void append_escaped(extra::string *es, csubstr val)
}
}
// flush the rest
this->append(val.sub(prev));
es->append(val.sub(prev));
#undef _c4flush_use_instead
}

24
test/test_suite/test_suite_event_handler.hpp
View File

@@ -7,9 +7,6 @@
#ifndef _C4_YML_EVENT_HANDLER_STACK_HPP_
#include "c4/yml/event_handler_stack.hpp"
#endif
#ifndef _C4_YML_STD_STRING_HPP_
#include "c4/yml/std/string.hpp"
#endif
#ifndef _C4_YML_DETAIL_PRINT_HPP_
#include "c4/yml/detail/print.hpp"
#endif
@@ -100,6 +97,7 @@ public:
m_has_yaml_directive = false;
for(TagDirective &td : m_tag_directives)
td = {};
m_val_buffers.clear();
m_val_buffers.resize((size_t)m_stack.size());
m_arena.clear();
m_arena.reserve(1024);
@@ -650,20 +648,20 @@ public:
EventSink& _buf_() noexcept
{
_RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)m_curr->level < m_val_buffers.size());
return m_val_buffers[(size_t)m_curr->level];
_RYML_CB_ASSERT(m_stack.m_callbacks, m_curr->level < m_val_buffers.size());
return m_val_buffers[m_curr->level];
}
EventSink& _buf_(id_type level) noexcept
{
_RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)level < m_val_buffers.size());
return m_val_buffers[(size_t)level];
_RYML_CB_ASSERT(m_stack.m_callbacks, level < m_val_buffers.size());
return m_val_buffers[level];
}
EventSink const& _buf_(id_type level) const noexcept
{
_RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)level < m_val_buffers.size());
return m_val_buffers[(size_t)level];
_RYML_CB_ASSERT(m_stack.m_callbacks, level < m_val_buffers.size());
return m_val_buffers[level];
}
static void _buf_flush_to_(EventSink &C4_RESTRICT src, EventSink &C4_RESTRICT dst) noexcept
@@ -686,8 +684,8 @@ public:
void _buf_ensure_(id_type size_needed) noexcept
{
if((size_t)size_needed > m_val_buffers.size())
m_val_buffers.resize((size_t)size_needed);
if(size_needed > m_val_buffers.size())
m_val_buffers.resize(size_needed);
}
C4_ALWAYS_INLINE void _send_(csubstr s) noexcept { _buf_().append(s); }
@@ -699,7 +697,7 @@ public:
_send_key_props_();
_send_(' ');
_send_(scalar_type_code);
_buf_().append_escaped(scalar);
append_escaped(&_buf_(), scalar);
_send_('\n');
}
void _send_val_scalar_(csubstr scalar, char scalar_type_code)
@@ -708,7 +706,7 @@ public:
_send_val_props_();
_send_(' ');
_send_(scalar_type_code);
_buf_().append_escaped(scalar);
append_escaped(&_buf_(), scalar);
_send_('\n');
}

13
test/test_tag_property.cpp
View File

@@ -1041,6 +1041,19 @@ TEST(tags, EHF6)
}
}
TEST(tags, fuzzcrash0)
{
Tree tree;
ExpectError::do_check(&tree, [&]{
parse_in_arena("%TAG !! " "\n"
"})" "\n"
"" "\n"
"!!<" "\n"
,
&tree);
});
}
//-----------------------------------------------------------------------------
//-----------------------------------------------------------------------------

20
tools/parse_emit.cpp
View File

@@ -7,7 +7,6 @@
#include <c4/yml/common.hpp>
#endif
#include <c4/fs/fs.hpp>
#include <c4/yml/detail/print.hpp>
#include <cstdio>
#include <chrono>
@@ -29,7 +28,6 @@ C4_SUPPRESS_WARNING_GCC("-Wuseless-cast")
bool quiet = false;
bool debug_tree = false;
bool emit_as_json = false;
bool timed_sections = false;
bool emit_to_string = false;
@@ -95,8 +93,6 @@ csubstr parse_args(int argc, const char *argv[])
timed_sections = true;
else if(arg == "-s" || arg == "--string")
emit_to_string = true;
else if(arg == "-d" || arg == "--debug")
debug_tree = true;
else if(arg == "-j" || arg == "--json")
emit_as_json = true;
else
@@ -214,22 +210,10 @@ void process_file(csubstr file)
TS(parse_yml);
yml::parse_in_place(file, to_substr(contents), &tree);
}
if(debug_tree)
{
TS(debug_tree);
yml::print_tree(tree);
}
if(emit_as_json)
{
{
TS(resolve_refs);
tree.resolve();
}
if(debug_tree)
{
TS(debug_resolved_tree);
yml::print_tree(tree);
}
TS(resolve_refs);
tree.resolve();
}
if(emit_to_string)
{

8
tools/yaml_events.cpp
View File

@@ -151,8 +151,6 @@ bool Args::parse(Args *args, int argc, const char *argv[])
return true;
}
C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
std::string load_file(csubstr filename)
{
if(filename == "-") // read from stdin
@@ -160,7 +158,7 @@ std::string load_file(csubstr filename)
std::string buf;
for(int c = std::getchar(); c != EOF; c = std::getchar())
{
buf.push_back((char)c);
buf.push_back(static_cast<char>(c));
if(buf.size() == buf.capacity())
buf.reserve(2u * (buf.capacity() >= 128u ? buf.capacity() : 128u));
}
@@ -187,12 +185,10 @@ void report_error(const char* msg, size_t length, Location loc, FILE *f)
if(loc.offset)
fprintf(f, " (%zuB):", loc.offset);
fputc(' ', f);
fprintf(f, "%.*s\n", (int)length, msg);
fprintf(f, "%.*s\n", static_cast<int>(length), msg);
fflush(f);
}
C4_SUPPRESS_WARNING_GCC_CLANG_POP
Callbacks create_custom_callbacks()
{
Callbacks callbacks = {};