minor fixes

README.md

@@ -432,37 +432,6 @@ CHECK(loc.col == 4u);
 
 ## Using ryml in your project
 
-### Package managers
-
-ryml is available in most package managers (thanks to all the
-contributors!) and linux distributions. But please be aware: those
-packages are maintained downstream of this repository, so if you have
-issues with the package, file a report with the respective maintainer.
-
-Here's a quick roundup (not maintained):
-* Package managers:
-  * [conan](https://conan.io/center/recipes/rapidyaml)
-  * [vcpkg](https://vcpkg.io/en/packages.html): `vcpkg install ryml`
-  * [PyPI](https://pypi.org/project/rapidyaml/)
-* Linux distributions:
-  * Arch Linux/Manjaro:
-    * [rapidyaml (aarch64)](https://archlinuxarm.org/packages/aarch64/rapidyaml)
-    * [rapidyaml-git (AUR)](https://aur.archlinux.org/packages/rapidyaml-git/)
-    * [python-rapidyaml-git (AUR)](https://aur.archlinux.org/packages/python-rapidyaml-git/)
-  * [Fedora Linux](https://getfedora.org/)/[EPEL](https://docs.fedoraproject.org/en-US/epel/):
-    * `dnf install rapidyaml-devel`
-    * `dnf install python3-rapidyaml`
-  * [Gentoo](https://packages.gentoo.org/packages/dev-cpp/rapidyaml)
-  * [OpenSuse](https://build.openbuildservice.org/package/show/Emulators/rapidyaml)
-  * [Slackbuilds](https://slackbuilds.org/repository/15.0/libraries/rapidyaml/)
-  * [AltLinux](https://packages.altlinux.org/en/sisyphus/srpms/rapidyaml/3006055151670528141)
-  
-Although package managers are very useful for quickly getting up to
-speed, the advised way is still to bring ryml as a submodule of your
-project, building both together. This makes it easy to track any
-upstream changes in ryml. Also, ryml is small and quick to build, so
-there's not much of a cost for building it with your project.
-
 ### Single header file
 ryml is provided chiefly as a cmake library project, but it can also
 be used as a single header file, and there is a [tool to
@@ -531,6 +500,38 @@ If you omit `--recursive`, after cloning you
 will have to do `git submodule update --init --recursive`
 to ensure ryml's submodules are checked out.
 
+### Package managers
+
+ryml is available in most package managers (thanks to all the
+contributors!) and linux distributions. But please be aware: those
+packages are maintained downstream of this repository, so if you have
+issues with the package, file a report with the respective maintainer.
+
+Here's a quick roundup (not maintained):
+* Package managers:
+  * [conan](https://conan.io/center/recipes/rapidyaml)
+  * [vcpkg](https://vcpkg.io/en/packages.html): `vcpkg install ryml`
+  * [PyPI](https://pypi.org/project/rapidyaml/)
+* Linux distributions:
+  * Arch Linux/Manjaro:
+    * [rapidyaml (aarch64)](https://archlinuxarm.org/packages/aarch64/rapidyaml)
+    * [rapidyaml-git (AUR)](https://aur.archlinux.org/packages/rapidyaml-git/)
+    * [python-rapidyaml-git (AUR)](https://aur.archlinux.org/packages/python-rapidyaml-git/)
+  * [Fedora Linux](https://getfedora.org/)/[EPEL](https://docs.fedoraproject.org/en-US/epel/):
+    * `dnf install rapidyaml-devel`
+    * `dnf install python3-rapidyaml`
+  * [Gentoo](https://packages.gentoo.org/packages/dev-cpp/rapidyaml)
+  * [OpenSuse](https://build.openbuildservice.org/package/show/Emulators/rapidyaml)
+  * [Slackbuilds](https://slackbuilds.org/repository/15.0/libraries/rapidyaml/)
+  * [AltLinux](https://packages.altlinux.org/en/sisyphus/srpms/rapidyaml/3006055151670528141)
+
+Although package managers are very useful for quickly getting up to
+speed, the advised way is still to bring ryml as a submodule of your
+project, building both together. This makes it easy to track any
+upstream changes in ryml. Also, ryml is small and quick to build, so
+there's not much of a cost for building it with your project.
+
+
 ### Quickstart samples
 
 These samples show different ways of getting ryml into your application. All the
@@ -555,6 +556,7 @@ more about each sample:
 | [`fetch_content`](./samples/fetch_content)      | **yes**                      | [`CMakeLists.txt`](./samples/fetch_content/CMakeLists.txt) | [`run.sh`](./samples/fetch_content/run.sh) |
 | [`find_package`](./samples/find_package)        | **no**<br>needs prior install or package  | [`CMakeLists.txt`](./samples/find_package/CMakeLists.txt) | [`run.sh`](./samples/find_package/run.sh) |
 
+
 ### CMake build settings for ryml
 The following cmake variables can be used to control the build behavior of
 ryml:
@@ -726,20 +728,16 @@ See also [the roadmap](./ROADMAP.md) for a list of future work.
 
 ### Known limitations
 
-ryml deliberately makes no effort to follow the standard in the
+ryml deliberately makes no effort to follow the YAML standard in the
 following situations:
 
-* ryml's tree does NOT accept containers are as mapping keys: keys
+* ryml's tree does NOT accept containers as map keys: keys stored in
-  must be scalars. HOWEVER, this is a limitation only of the tree. The
+  the tree must always be scalars. HOWEVER, this is a limitation only
-  event-based parser engine DOES parse container keys. The parser
+  of the final tree. The event-based parse engine DOES parse container
-  engine is the result of a recent refactor and its usage is meant to
+  keys, as it is is meant to be used by other programming languages to
-  be used by other programming languages to create their native
+  create their native data-structures, and it is fully tested and
-  data-structures. This engine is fully tested and fully conformant
+  fully conformant (other than the general error permissiveness noted
-  (other than the general error permissiveness noted below). But
+  below).
-  because it is recent, it is still undocumented, and it requires some
-  API cleanup before being ready for isolated use. Please get in touch
-  if you are interested in integrating the event-based parser engine
-  without the standalone `ryml::parse_*()`
 * Tab characters after `:` and `-` are not accepted tokens, unless
   ryml is compiled with the macro `RYML_WITH_TAB_TOKENS`. This
   requirement exists because checking for tabs introduces branching
@@ -774,11 +772,11 @@ following situations:
 If you do run into trouble and would like to investigate conformance
 of your YAML code, **beware** of existing online YAML linters, many of
 which are not fully conformant. Instead, try using
-[https://play.yaml.io](https://play.yaml.io), an amazing tool which
+[https://play.yaml.io](https://play.yaml.io), an amazingly useful tool
-lets you dynamically input your YAML and continuously see the results
+which lets you dynamically input your YAML and continuously see the
-from all the existing parsers (kudos to @ingydotnet and the people
+results from all the existing parsers (kudos to @ingydotnet and the
-from the YAML test suite). And of course, if you detect anything wrong
+people from the YAML test suite). And of course, if you detect
-with ryml, please [open an
+anything wrong with ryml, please [open an
 issue](https://github.com/biojppm/rapidyaml/issues) so that we can
 improve.
 

ROADMAP.md

@@ -1,2 +1,2 @@
 
-Moved! See ryml's [Kanban board on github](https://github.com/biojppm/rapidyaml/projects/1).
+Moved! See rapidyaml's [Kanban board on github](https://github.com/users/biojppm/projects/1/views/1).

doc/index.rst

@@ -35,10 +35,10 @@ ryml is written in C++11, and compiles cleanly with:
 
 * Visual Studio 2015 and later
 
-* clang++ 3.9 and later
-
 * g++ 4.8 and later
 
+* clang++ 3.9 and later
+
 * Intel Compiler
 
  .. note::

doc/sphinx_quicklinks.rst

@@ -13,7 +13,7 @@ Quick links
 
   * `Pull Requests <https://github.com/biojppm/rapidyaml/pull>`_
 
-  * `Kanban board <https://github.com/biojppm/rapidyaml/projects/1>`_
+  * `Kanban board <https://github.com/users/biojppm/projects/1/views/1>`_
 
 * Latest release: `0.6.0 <https://github.com/biojppm/rapidyaml/releases/tag/v0.6.0>`_
 

doc/sphinx_yaml_standard.rst

@@ -17,7 +17,7 @@ welcome.
    linters**, many of which are not fully conformant; instead, try
    using `https://play.yaml.io
    <https://play.yaml.io/main/parser?input=IyBFZGl0IE1lIQoKJVlBTUwgMS4yCi0tLQpmb286IEhlbGxvLCBZQU1MIQpiYXI6IFsxMjMsIHRydWVdCmJhejoKLSBvbmUKLSB0d28KLSBudWxsCg==>`__,
-   an amazing tool which lets you dynamically input your YAML and
+   an amazingly useful tool which lets you dynamically input your YAML and
    continuously see the results from all the existing parsers (kudos
    to @ingydotnet and the people from the YAML test suite). And of
    course, if you detect anything wrong with ryml, please `open an
@@ -31,17 +31,13 @@ Deliberate deviations
 ryml deliberately makes no effort to follow the standard in the
 following situations:
 
--  ryml's tree does NOT accept containers are as mapping keys: keys
+-  ryml's tree does NOT accept containers as map keys: keys stored in
-   must be scalars. HOWEVER, this is a limitation only of the tree. The
+   the tree must always be scalars. HOWEVER, this is a limitation only
-   event-based parser engine DOES parse container keys. The parser
+   of the final tree. The event-based parse engine DOES parse container
-   engine is the result of a recent refactor and its usage is meant to
+   keys, as it is is meant to be used by other programming languages to
-   be used by other programming languages to create their native
+   create their native data-structures, and it is fully tested and
-   data-structures. This engine is fully tested and fully conformant
+   fully conformant (other than the general error permissiveness noted
-   (other than the general error permissiveness noted below). But
+   below).
-   because it is recent, it is still undocumented, and it requires some
-   API cleanup before being ready for isolated use. Please get in touch
-   if you are interested in integrating the event-based parser engine
-   without the standalone `ryml::parse_*()`
 -  Tab characters after ``:`` and ``-`` are not accepted tokens, unless
    ryml is compiled with the macro ``RYML_WITH_TAB_TOKENS``. This
    requirement exists because checking for tabs introduces branching

src/c4/yml/detail/stack.hpp

@@ -34,11 +34,11 @@ public:
 
 public:
 
-    T         m_buf[size_t(N)];
+    T              m_buf[size_t(N)];
-    T *       m_stack;
+    T *C4_RESTRICT m_stack;
-    id_type   m_size;
+    id_type        m_size;
-    id_type   m_capacity;
+    id_type        m_capacity;
-    Callbacks m_callbacks;
+    Callbacks      m_callbacks;
 
 public:
 

src/c4/yml/event_handler_stack.hpp

@@ -46,7 +46,7 @@ public:
     state *C4_RESTRICT   m_curr;    ///< current stack level: top of the stack. cached here for easier access.
     state *C4_RESTRICT   m_parent;  ///< parent of the current stack level.
     pfn_relocate_arena   m_relocate_arena; ///< callback when the arena gets relocated
-    void *C4_RESTRICT    m_relocate_arena_data;
+    void *               m_relocate_arena_data;
 
 protected:
 

test/test_parse_engine_6_qmrk.cpp

@@ -292,6 +292,32 @@ ENGINE_TEST(Qmrk3,
     ___(ps.end_stream());
 }
 
+ENGINE_TEST(Qmrk4_0,
+            ("[?baz:,]",
+             "[{?baz: }]"),
+            "+STR\n"
+            "+DOC\n"
+            "+SEQ []\n"
+            "+MAP {}\n"
+            "=VAL :?baz\n"
+            "=VAL :\n"
+            "-MAP\n"
+            "-SEQ\n"
+            "-DOC\n"
+            "-STR\n")
+{
+    ___(ps.begin_stream());
+    ___(ps.begin_doc());
+    ___(ps.begin_seq_val_flow());
+    ___(ps.begin_map_val_flow());
+    ___(ps.set_key_scalar_plain("?baz"));
+    ___(ps.set_val_scalar_plain({}));
+    ___(ps.end_map());
+    ___(ps.end_seq());
+    ___(ps.end_doc());
+    ___(ps.end_stream());
+}
+
 ENGINE_TEST(Qmrk4,
             ("[ ? an explicit key, ? foo,? bar,?baz:,?bat]",
              "[{an explicit key: },{foo: },{bar: },{?baz: },?bat]"),

test/test_suite/test_suite_event_handler.cpp

@@ -72,7 +72,7 @@ void append_escaped(extra::string *es, csubstr val)
         }
     }
     // flush the rest
-    this->append(val.sub(prev));
+    es->append(val.sub(prev));
     #undef _c4flush_use_instead
 }
 

test/test_suite/test_suite_event_handler.hpp

@@ -7,9 +7,6 @@
 #ifndef _C4_YML_EVENT_HANDLER_STACK_HPP_
 #include "c4/yml/event_handler_stack.hpp"
 #endif
-#ifndef _C4_YML_STD_STRING_HPP_
-#include "c4/yml/std/string.hpp"
-#endif
 #ifndef _C4_YML_DETAIL_PRINT_HPP_
 #include "c4/yml/detail/print.hpp"
 #endif
@@ -100,6 +97,7 @@ public:
         m_has_yaml_directive = false;
         for(TagDirective &td : m_tag_directives)
             td = {};
+        m_val_buffers.clear();
         m_val_buffers.resize((size_t)m_stack.size());
         m_arena.clear();
         m_arena.reserve(1024);
@@ -650,20 +648,20 @@ public:
 
     EventSink& _buf_() noexcept
     {
-        _RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)m_curr->level < m_val_buffers.size());
+        _RYML_CB_ASSERT(m_stack.m_callbacks, m_curr->level < m_val_buffers.size());
-        return m_val_buffers[(size_t)m_curr->level];
+        return m_val_buffers[m_curr->level];
     }
 
     EventSink& _buf_(id_type level) noexcept
     {
-        _RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)level < m_val_buffers.size());
+        _RYML_CB_ASSERT(m_stack.m_callbacks, level < m_val_buffers.size());
-        return m_val_buffers[(size_t)level];
+        return m_val_buffers[level];
     }
 
     EventSink const& _buf_(id_type level) const noexcept
     {
-        _RYML_CB_ASSERT(m_stack.m_callbacks, (size_t)level < m_val_buffers.size());
+        _RYML_CB_ASSERT(m_stack.m_callbacks, level < m_val_buffers.size());
-        return m_val_buffers[(size_t)level];
+        return m_val_buffers[level];
     }
 
     static void _buf_flush_to_(EventSink &C4_RESTRICT src, EventSink &C4_RESTRICT dst) noexcept
@@ -686,8 +684,8 @@ public:
 
     void _buf_ensure_(id_type size_needed) noexcept
     {
-        if((size_t)size_needed > m_val_buffers.size())
+        if(size_needed > m_val_buffers.size())
-            m_val_buffers.resize((size_t)size_needed);
+            m_val_buffers.resize(size_needed);
     }
 
     C4_ALWAYS_INLINE void _send_(csubstr s) noexcept { _buf_().append(s); }
@@ -699,7 +697,7 @@ public:
         _send_key_props_();
         _send_(' ');
         _send_(scalar_type_code);
-        _buf_().append_escaped(scalar);
+        append_escaped(&_buf_(), scalar);
         _send_('\n');
     }
     void _send_val_scalar_(csubstr scalar, char scalar_type_code)
@@ -708,7 +706,7 @@ public:
         _send_val_props_();
         _send_(' ');
         _send_(scalar_type_code);
-        _buf_().append_escaped(scalar);
+        append_escaped(&_buf_(), scalar);
         _send_('\n');
     }
 

test/test_tag_property.cpp

@@ -1041,6 +1041,19 @@ TEST(tags, EHF6)
     }
 }
 
+TEST(tags, fuzzcrash0)
+{
+    Tree tree;
+    ExpectError::do_check(&tree, [&]{
+        parse_in_arena("%TAG !! " "\n"
+                       "})"       "\n"
+                       ""         "\n"
+                       "!!<"      "\n"
+                       ,
+                       &tree);
+    });
+}
+
 
 //-----------------------------------------------------------------------------
 //-----------------------------------------------------------------------------

tools/parse_emit.cpp

@@ -7,7 +7,6 @@
 #include <c4/yml/common.hpp>
 #endif
 #include <c4/fs/fs.hpp>
-#include <c4/yml/detail/print.hpp>
 
 #include <cstdio>
 #include <chrono>
@@ -29,7 +28,6 @@ C4_SUPPRESS_WARNING_GCC("-Wuseless-cast")
 
 
 bool quiet = false;
-bool debug_tree = false;
 bool emit_as_json = false;
 bool timed_sections = false;
 bool emit_to_string = false;
@@ -95,8 +93,6 @@ csubstr parse_args(int argc, const char *argv[])
             timed_sections = true;
         else if(arg == "-s" || arg == "--string")
             emit_to_string = true;
-        else if(arg == "-d" || arg == "--debug")
-            debug_tree = true;
         else if(arg == "-j" || arg == "--json")
             emit_as_json = true;
         else
@@ -214,22 +210,10 @@ void process_file(csubstr file)
         TS(parse_yml);
         yml::parse_in_place(file, to_substr(contents), &tree);
     }
-    if(debug_tree)
-    {
-        TS(debug_tree);
-        yml::print_tree(tree);
-    }
     if(emit_as_json)
     {
-        {
+        TS(resolve_refs);
-            TS(resolve_refs);
+        tree.resolve();
-            tree.resolve();
-        }
-        if(debug_tree)
-        {
-            TS(debug_resolved_tree);
-            yml::print_tree(tree);
-        }
     }
     if(emit_to_string)
     {

tools/yaml_events.cpp

@@ -151,8 +151,6 @@ bool Args::parse(Args *args, int argc, const char *argv[])
     return true;
 }
 
-C4_SUPPRESS_WARNING_GCC_CLANG_WITH_PUSH("-Wold-style-cast")
-
 std::string load_file(csubstr filename)
 {
     if(filename == "-") // read from stdin
@@ -160,7 +158,7 @@ std::string load_file(csubstr filename)
         std::string buf;
         for(int c = std::getchar(); c != EOF; c = std::getchar())
         {
-            buf.push_back((char)c);
+            buf.push_back(static_cast<char>(c));
             if(buf.size() == buf.capacity())
                 buf.reserve(2u * (buf.capacity() >= 128u ? buf.capacity() : 128u));
         }
@@ -187,12 +185,10 @@ void report_error(const char* msg, size_t length, Location loc, FILE *f)
     if(loc.offset)
         fprintf(f, " (%zuB):", loc.offset);
     fputc(' ', f);
-    fprintf(f, "%.*s\n", (int)length, msg);
+    fprintf(f, "%.*s\n", static_cast<int>(length), msg);
     fflush(f);
 }
 
-C4_SUPPRESS_WARNING_GCC_CLANG_POP
-
 Callbacks create_custom_callbacks()
 {
     Callbacks callbacks = {};