This post is a follow up to my (kinda lengthy) deep dive into what I thought was a bug in golang’s
Instead, I realized that generally speaking,
json.Decoder can be misunderstood - which may lead to unintended consequences. In this post, I will demonstrate a safer pattern that ought to be used instead of the prevailing wisdom.
Googling: “json.decoder example golang”
I ran a few google searches queries using some permutation of the following:
json.decoder example golang
The results were your standard mix of documentation from
golang.org, blog posts on
medium.com/random mom&pop devsites (like this one!), and a few
Fortunately, results from
golang.org are highly ranked - while it may be a bit harder to parse through golang's src code, the documentation is fairly thorough and more importantly: correct. (Personally, I would have preferred some additional context in the docs expounding on some of the gotchas I discuss on my other post but I digress)
Some of the threads I observed in Stack Overflow that referenced
json.Decoder pulled directly from the docs (and therefore were also correct). Other's (probably) pulled from medium/other blog post sites similar to the ones I found googling around and were inaccurate/advocating incorrect usage.
On the surfact, this code looks sound. I forklifted the src into go playground and ran it.
…It works! Great. But, what happens if we fubar the JSON string?
This code should not work at all! Our JSON string is clearly malformed and we expect — based on the logic — the code to panic.
This is the entire issue in a nutshell. I expound in detail in my other post but in one (kinda long) sentence:
json.Decoder.Decodewas implemented for parsing streaming JSON data, meaning it will always traverse the JSON string until it finds a satisfactory, closing bracket (I use the term satisfactory here because it does use a stack to keep track of inner brackets).
So, in order to detect the malformed json, we must actually run this logic in a loop — like so:
Note the key diff here:
From the golang docs, this example says it best:
In this usage, we create a new
json.Decoder instance from the
NewDecoder method and then continually loop and try to decode a chunk of our JSON string until we detect the end of the string (sucessfully breaking out of the loop) or an error.
I would take this a step further and only prefer to use
json.Decoder when I am specifically working with streaming JSON data.
At any rate, this the The Way. Please keep this in mind going forward should you choose to use
json.Decoder for your JSON string parsing needs.
A few notes
- A big caveat I’d like to point out is that I did not perform any rigorous analysis of golang examples in the wild when I was inspired to write this post. My analysis comes largely from anecdotal examples I observed while researching my other deep dive article about the nature of
- I certianly am not being critical of the medium/mom&pop blogs I linked to earlier, instead I am merely pointing out that it is very easy to assume that the trivial usage of
json.Decoder.Decodepresented on those posts is generally correct when in actuality this is not the case.
- I think someone (maybe me? perhaps you?) ought to open a PR against godocs to clarify gotchas/differences between
PS: this article is x-posted from my dev blog
Published with this awesome tool.