JSON for Modern C++ version 3.10.0
nikki93 2021-08-19 05:06:54 +0000 UTC [ - ]
The cJSON's C API or rapidjson's somewhat less ergonomic API has felt fine because I usually have read / write be reflective and don't write that much manual JSON code.
Specifically for the case where compile time doesn't matter and I need expressive JSON object mutation in C++ (or some tradeoff thereof), I think this library seems good.
Another big feature (which I feel like it could be louder about -- it really is a big one!) is that you can also generate binary data in a bunch of formats (BSON, CBOR, MessagePack and UJBJSON) that have a node structure similar to JSON's -- from the same in-memory instances of this library's data type. That sort of thing is something I've desired for various reasons (smaller file types with potentially better network send time for asynchronous sending / downloads (for real-time multiplayer you still basically want to encode to something that doesn't embed field string names)). I do think I may end up doing it at one layer above in the engine level and just have a different backend other than cJSON etc. too though...
qalmakka 2021-08-19 09:11:14 +0000 UTC [ - ]
Ironically, cJSON is _worse_ in my use case due to it not supporting allocators at all, as you wrote. Nlohmann fully supports C++ allocators, so it's trivial to allocate on SPIRAM instead of the limited DRAM of the ESP32. Support for allocators is why I often tend to pick C++ for projects with heterogeneous kinds of memory.
Also, Nlohmann/JSON supports strong typed serialization and deserialization, which in my experience vastly reduces the amount of errors people tend to make when reading/writing data to JSON. I've found simpler to rewrite some critical code that was parsing very complex data using cJSON to Nlohmann/JSON than fixing up all the tiny errors the previous developer made while reading "raw" JSON, parsing numbers as enums, strings as other kinds of enums, and so on.
cibomahto 2021-08-19 12:17:49 +0000 UTC [ - ]
Here's what I use on ESP32 to push allocation to SPIRAM: https://gist.github.com/cibomahto/a29b6662847e13c61b47a194fa...
nlohmann 2021-08-19 11:57:57 +0000 UTC [ - ]
ephaeton 2021-08-19 05:19:52 +0000 UTC [ - ]
Minusses: although I'd like a way to append a (C-)array into a JSON-array "at once" and not iteratively (i.e., O(n) instead of O(n log n)). Also, lack of support for JSON schema is .. slightly annoying.
nlohmann 2021-08-19 11:56:23 +0000 UTC [ - ]
AlexandrB 2021-08-19 12:26:30 +0000 UTC [ - ]
nikki93 2021-08-19 05:25:16 +0000 UTC [ - ]
I didn't realize that about the array complexity. Can you not just initialize a JSON array of N elements in O(N) (conceding a `.reserve(N)` style call beforehand if required)? rapidjson is pretty good about that sort of thing, cJSON's arrays are linked list so I basically think of its performance as at a different level and it's mostly about compile time for me.
ephaeton 2021-08-19 05:37:36 +0000 UTC [ - ]
esprehn 2021-08-19 03:49:46 +0000 UTC [ - ]
It seems they're targeting C++11?
nlohmann 2021-08-19 12:01:59 +0000 UTC [ - ]
The string_view lookup is nearly done, but I did not want to wait for it, because it would have delayed the release even more.
I'm also working on supporting unordered_map - using it as container for objects would be easy if we would just break the existing API - the hard part is to support it with the current (probably bad designed) template API.
MauranKilom 2021-08-19 08:11:12 +0000 UTC [ - ]
std::unordered_map has massive overhead in space and time. Maybe I misunderstand your idea, but outside of extremely niche use cases, std::unordered_map is basically never the right tool (unless you don't care in the slightest about performance or memory overhead).
agent327 2021-08-19 08:36:08 +0000 UTC [ - ]
einpoklum 2021-08-19 10:57:01 +0000 UTC [ - ]
https://stackoverflow.com/a/42588384/1593077
and the link there. Not sure that's what GP meant though.
agent327 2021-08-19 13:15:01 +0000 UTC [ - ]
And I don't like it when people use ridiculous hyperbole to describe what is in reality barely perceptible overhead, so it would be good if the person I originally responded to came out and defended his POV. Hopefully using actual numbers instead of agitated handwaving and screaming...
MauranKilom 2021-08-19 16:06:10 +0000 UTC [ - ]
Pretty sure that makes it impossible to implement merge without pointer/iterator invalidation, which is a constraint imposed by the standard (https://en.cppreference.com/w/cpp/container/unordered_map/me...).
> it would be good if the person I originally responded to came out and defended his POV
See https://probablydance.com/2017/02/26/i-wrote-the-fastest-has... if you want numbers, or for example this talk by the same author https://www.youtube.com/watch?v=M2fKMP47slQ.
beached_whale 2021-08-19 15:49:05 +0000 UTC [ - ]
The issue is that the implementors won't change their implementation of unordered_map as it's an ABI break, to say it simply. It could be better. But also, there are other tools like flat maps/open addressing that are not done in the std library.
wmkn 2021-08-19 12:41:56 +0000 UTC [ - ]
Not sure what your rationale is for saying it’s only useful for extremely niche use cases. I would be curious to know why you think so.
MauranKilom 2021-08-19 16:17:19 +0000 UTC [ - ]
See https://www.youtube.com/watch?v=M2fKMP47slQ for a more comprehensive discussion, or the sibling threads here.
spacechild1 2021-08-19 10:29:57 +0000 UTC [ - ]
std::map is only useful if you need the data to be ordered, otherwise std::unordered_map is the right default choice.
> std::unordered_map has massive overhead in space and time.
Yes, std::unordered_map uses a bit more memory, but please show me some benchmarks where it is slower than std::map - especially with string keys!
BTW, for very small numbers of items the most efficient solution is often a plain std::vector + linear search.
MauranKilom 2021-08-19 16:14:07 +0000 UTC [ - ]
That's sometimes ok, particularly if you don't care about performance (beyond asymptotic behavior). But if you do, you'd probably be well-advised to try out hash map implementations that actually don't leave performance on the table by design.
adzm 2021-08-19 12:08:14 +0000 UTC [ - ]
This is why flat_map is a great solution. It's basically an ordered vector with a map-like interface, but without the memory allocation (and cache locality issues) for each node. Until you start getting into thousands of elements it is probably the best choice of container.
codr7 2021-08-19 11:40:19 +0000 UTC [ - ]
And ordered maps fall somewhere in the middle.
They might make sense if you're creating massive amounts of medium sized maps.
Or if you need range searches etc.
It's all compromises, there are no hard rules.
batty_alex 2021-08-19 11:44:45 +0000 UTC [ - ]
std::unordered_map is typically the go-to hash map on any given C++ day. You’re probably fine with std::map outside of large n
EDIT: being pedantic
adzm 2021-08-19 12:03:12 +0000 UTC [ - ]
beached_whale 2021-08-19 04:25:42 +0000 UTC [ - ]
quietbritishjim 2021-08-19 10:58:57 +0000 UTC [ - ]
nlohmann 2021-08-19 12:03:30 +0000 UTC [ - ]
But to be honest, I have not yet played around with PMR.
beached_whale 2021-08-19 12:50:25 +0000 UTC [ - ]
pjmlp 2021-08-19 05:13:42 +0000 UTC [ - ]
beached_whale 2021-08-19 05:34:40 +0000 UTC [ - ]
pjmlp 2021-08-19 05:38:52 +0000 UTC [ - ]
For example, C++/WinRT is a C++17 library, yet plenty of samples use C style strings and vectors.
howolduis 2021-08-19 04:17:29 +0000 UTC [ - ]
rualca 2021-08-19 05:51:07 +0000 UTC [ - ]
Do you find it unreasonable to point out that what's been advertised doesn't match what's being offered?
nlohmann 2021-08-19 12:04:27 +0000 UTC [ - ]
rualca 2021-08-19 12:08:42 +0000 UTC [ - ]
nlohmann 2021-08-19 12:13:01 +0000 UTC [ - ]
maleldil 2021-08-19 12:22:52 +0000 UTC [ - ]
bobnamob 2021-08-19 12:21:10 +0000 UTC [ - ]
q-rews 2021-08-19 05:18:47 +0000 UTC [ - ]
pjmlp 2021-08-19 05:12:55 +0000 UTC [ - ]
mike_hock 2021-08-19 08:29:45 +0000 UTC [ - ]
microtherion 2021-08-19 13:02:23 +0000 UTC [ - ]
SloopJon 2021-08-19 05:14:24 +0000 UTC [ - ]
There are lots of other libraries, and surely some faster ones, but I haven't felt any need to change. I was nervous when I got an inexplicable overload ambiguity in xlclang++ for AIX, but it went away when I grabbed a newer version of the library.
iamvnraju 2021-08-19 05:39:53 +0000 UTC [ - ]
_3u10 2021-08-19 04:31:29 +0000 UTC [ - ]
https://github.com/simdjson/simdjson/blob/master/doc/basics....
adzm 2021-08-19 12:10:57 +0000 UTC [ - ]
joshxyz 2021-08-19 08:09:14 +0000 UTC [ - ]
these projects are just commendable!
modeless 2021-08-19 06:55:29 +0000 UTC [ - ]
jcelerier 2021-08-19 07:00:31 +0000 UTC [ - ]
modeless 2021-08-19 14:28:34 +0000 UTC [ - ]
tsbinz 2021-08-19 07:25:26 +0000 UTC [ - ]
modeless 2021-08-19 14:23:52 +0000 UTC [ - ]
versatran01 2021-08-19 03:58:49 +0000 UTC [ - ]
darknavi 2021-08-19 05:27:59 +0000 UTC [ - ]
chrisjericho 2021-08-19 05:54:24 +0000 UTC [ - ]
IIRC, you would need to modify the statement to "find_package(Boost REQUIRED COMPONENTS filesystem)" if you want to use boost dependencies that need a separate .dll/.so (in this case boost::filesystem). However I rarely need to use these, so I may be wrong.
electroly 2021-08-19 15:37:41 +0000 UTC [ - ]
Johnnynator 2021-08-19 05:59:34 +0000 UTC [ - ]
It gets far more complicated with C++11, since you also need a ton of other boost modules there.
For more Details you can read the Readme of it. https://github.com/boostorg/json
nly 2021-08-19 05:55:35 +0000 UTC [ - ]
With proper CI it's not a big deal
jcelerier 2021-08-19 07:04:53 +0000 UTC [ - ]
wget boost_1_77.tar.gz
tar xaf boost*
cmake ... -DBOOST_ROOT=.../boost_1_77
Works for the huge majority of boost ; most parts that required a build were the parts that were integrated in c++ like thread, regex, chrono, filesystem
geokon 2021-08-19 06:16:59 +0000 UTC [ - ]
https://github.com/cpp-pm/hunter
There is a bit of a learning curve, but it's the only dependency manager that does things right (all from within CMake)
rualca 2021-08-19 05:56:52 +0000 UTC [ - ]
beached_whale 2021-08-19 04:07:26 +0000 UTC [ - ]
newusertoday 2021-08-19 06:30:29 +0000 UTC [ - ]
einpoklum 2021-08-19 10:54:45 +0000 UTC [ - ]
> Speed. There are certainly faster JSON libraries out there. However, if your goal is to speed up your development by adding JSON support with a single header, then this library is the way to go.
It is essentially admitting to being slow(er). We are not supposed to pay for code modernity with performance. In fact, it should be the other way around.
Does someone know the innards of this library as opposed to, say, jsoncpp or other popular libraries, and describe the differences?
nlohmann 2021-08-19 12:07:11 +0000 UTC [ - ]
einpoklum 2021-08-19 16:12:59 +0000 UTC [ - ]
You're making the false assumption that there is a compromise to be made - but there isn't. A modern JSON library should be the fastest (or about the same speed as the fastest), and vice-versa.
TeeMassive 2021-08-19 04:45:00 +0000 UTC [ - ]
So what? The format is so simple that the inconsistencies doesn't even matter and hell the format can be inferred from the data and that can even be automated. Unlike JSON/XML, most software can automatically make sense of CSV; JSON needs a developer to understand it to transform it into workable data.
> CSVs often begin life as exported spreadsheets or table dumps from legacy databases, and often end life as a pile of undifferentiated files in a data lake, awaiting the restoration of their precious metadata so they can be organized and mined for insights.
CSVs are the de facto way to export spreadsheets and if you have to do this then the problem are that you are still using spreadsheets in your data flow, not CSVs. Same goes for legacy systems.
Also CSVs comes from places where having more sophisticated data serializers is near impossible and the time to code, compute, parse and analyze CSVs is trivial compared to any other format out there.
> I’ve spent many years battling CSVs in various capacities. As an engineer at Trifacta, a leading data preparation tool vendor, I saw numerous customers struggling under the burden of myriad CSVs that were the product of even more numerous Excel spreadsheet exports and legacy database dumps. As an engineering leader in biotech, my team struggled to ingest CSVs from research institutions and hospital networks who gave us sample data as CSVs more often than not.
Of course if you are working in biotech then you'll work with scientists (horrible coders most of the time) and legacy systems because it's a domain that's very adverse to change that might break stuff. And it's not the CSV format's fault if it's used as the wrong tool for the job.
> Another big drawback of CSV is its almost complete lack of metadata.
Again, emphasis on the right tool for the job. For simple tasks metada are not needed.
Stratoscope 2021-08-19 04:55:00 +0000 UTC [ - ]
https://news.ycombinator.com/item?id=28221654
Your quote "The CSV format itself is notoriously inconsistent" comes from this page which the thread discusses:
knorker 2021-08-19 08:25:15 +0000 UTC [ - ]
The second paragraph talks about setting a #define before including a header, as if this is still the 90's.
nlohmann 2021-08-19 12:08:56 +0000 UTC [ - ]
mike_hock 2021-08-19 08:49:49 +0000 UTC [ - ]
knorker 2021-08-19 11:22:36 +0000 UTC [ - ]
It can be a trade-off, but becoming less and less so. I'll admit to not looking at the implementation, but when you sell it as "modern" then it's not OK to do it for performance, for example.
And almost every single use I've seen "for performance" actually has zero or negligible performance impact.
a-dub 2021-08-19 10:27:25 +0000 UTC [ - ]
einpoklum 2021-08-19 11:00:31 +0000 UTC [ - ]
Moreover, because of `#define`s, the effect of including a file can be entirely different, so a compiler can never say "Oh, I know what's in this include file already, I don't need to include it again in other translation units".
Of course #define's have many uses, and at present cannot be avoided entirely, but C++ has been making an effort to have other mechanisms put in place so as to obviate the use of #define's
In particular, look for CppCon talks about Modules.
quietbritishjim 2021-08-19 11:32:00 +0000 UTC [ - ]
lazypenguin 2021-08-19 12:20:04 +0000 UTC [ - ]
Dear @nlohmann and contributors. Thank you for this library, I’ve used it in the past and appreciate your work and efforts.