Parsing JSON

JSON is a very simple data format. It’s widely used, and Nim’s standard library has sup-port for both parsing and generating it. This makes JSON a good candidate for storing the two message fields.

A typical JSON object contains multiple fields. The field names are simple quoted strings, and the values can be integers, floats, strings, arrays, or other objects.

1 In particular, C++ and Java use the ^public and ^private keywords to denote the visibility of identifiers.

Listing 3.6 Message type definition and proc stub

Defines a new Message type. The * export marker is placed after the name of the type.

Field definitions follow the type definition and are exported in a similar way.

Defines a new parseMessage procedure. The export marker is also used to export it.

The discard is necessary because the body of a procedure can’t be empty.

Let’s look back to the conversation about Game of Thrones in listing 3.1. One of the first messages that I sent was, “What did you guys think about the latest Game of Thrones episode?” This can be represented using JSON like so.

{

"username": "Dominik",

"message": "What did you guys think about the latest Game of Thrones episode?"

}

Parsing JSON is very easy in Nim. First, import the json module by adding import json to the top of your file. Then, replace the discard statement in the parseMessage proc with let dataJson = parseJson(data). The next listing shows the protocol module with the additions in bold.

import json type

Message* = object username*: string message*: string

proc parseMessage*(data: string): Message = let dataJson = parseJson(data)

The parseJson procedure defined in the json module accepts a string and returns a value of the JsonNode type.

JsonNode is a variant type. This means that which fields in the object can be accessed is determined by the value of one or more other fields that are always defined in that type. In the case of JsonNode, the kind field determines the kind of JSON node that was parsed.

There are seven different kinds of JSON values. The JsonNodeKind type is an enum with a value for each kind of JSON value. The following listing shows a list of various JSON values together with the JsonNodeKind types that they map to.

import json

assert parseJson("null").kind == JNull assert parseJson("true").kind == JBool assert parseJson("42").kind == JInt assert parseJson("3.14").kind == JFloat assert parseJson("\"Hi\"").kind == JString

assert parseJson("""{ "key": "value" }""").kind == JObject assert parseJson("[1, 2, 3, 4]").kind == JArray

Listing 3.7 A representation of a message in JSON

Listing 3.8 Parsing JSON in protocol.nim

Listing 3.9 The mapping between JSON values and the JsonNodeKind type The curly brackets define an object. The username field with

the corresponding value

The message field with the corresponding value

When you’re parsing arbitrary JSON data, a variant type is required because the com-piler has no way of knowing at compile time what the resulting JSON type should be.

The type is only known at runtime. This is why the parseJson procedure returns a JsonNode type whose contents differ depending on the kind of JSON value that was passed into it.

The last two JSON values shown in listing 3.9 are collections. The JObject kind rep-resents a mapping between a string and a JsonNode. The JArray kind stores a list of JsonNodes.

You can access the fields of a JObject by using the [] operator. It’s similar to the array and sequence [] operator but takes a string as its argument. The string determines the field whose value you want to retrieve from the JObject. The [] oper-ator returns a JsonNode value.

A little information about variant types

A variant type is an object type whose fields change depending on the value of one or more fields. An example will make this clearer:

type

var obj = Box(empty: false, contents: "Hello") assert obj.contents == "Hello"

var obj2 = Box(empty: true) echo(obj2.contents)

The preceding code shows how an ordinary box that may be empty can be modeled.

The end of the listing shows an erroneous case where the contents of an empty box are accessed. It should be no surprise that compiling and running that code will result in an error:

Traceback (most recent call last)

variant.nim(13) variant

system.nim(2533) sysFatal

Error: unhandled exception: contents is not accessible [FieldError]

This is a very simple variant type with only two states. You can also use enum types in the case statement of a variant type. This is common and is used in the Json-Node type.

A variant type is defined much

like other object types. The difference is the case statement under the definition of the object. This defines an empty field in this type.

If the empty field is false, the fields defined under this branch will be accessible.

The contents field will be accessible if empty == false.

No additional fields are defined if empty == true.

When the empty field is set to false in the constructor, the contents field can also be specified.

Because obj.empty is false, the contents field can be accessed.

This will result in an error because the contents field can’t be accessed, because empty is true.

import json

WARNING: THE KIND MATTERS Calling the [] operator with a string on a Json-Node whose kind field isn’t JObject will result in an exception being raised.

So, how can you retrieve the username field from the parsed JsonNode? Simply using dataJson["username"] will return another JsonNode, unless the username field doesn’t exist in the parsed JObject, in which case a KeyError exception will be raised.

In the preceding code, the JsonNode kind that dataJson["username"] returns is JString because that field holds a string value, so you can retrieve the string value using the getStr procedure. There’s a get procedure for each of the JsonNode kinds, and each get procedure will return a default value if the type of the value it’s meant to be returning doesn’t match the JsonNode kind.

THE DEFAULT VALUE FOR GET PROCEDURES The default value returned by the get procedures can be overridden. To override, pass the value you want to be returned by default as the second argument to the procedure; for example, node.getStr("Bad kind").

Once you have the username, you can assign it to a new instance of the Message type.

The next listing shows the full protocol module with the newly added assignments in bold.

proc parseMessage*(data: string): Message = let dataJson = parseJson(data)

result.username = dataJson["username"].getStr() result.message = dataJson["message"].getStr()

Just add two lines of code, and you’re done.

Listing 3.10 Assigning parsed data to the result variable Parses the data string and returns a JsonNode type, which is then assigned to the obj variable

The returned JsonNode has a JObject kind because that’s the kind of the JSON contained in the data string.

Fields are accessed using the [] operator.

It returns another JsonNode, and in this case its kind is a JString.

Because the [] operator returns a JsonNode, the value that it contains must be accessed explicitly via the field that contains it. In JString’s case, this is str. Generally you’re better off using the getStr proc.

Gets the value under the

"username" key and assigns its string value to the username field of the resulting Message

Does the same here, but instead gets the value under the

"message" key

You should test your code as quickly and as often as you can. You could do so now by starting to integrate your new module with the client module, but it’s much better to test code as separate units. The protocol module is a good unit of code to test in isolation.

When testing a module, it’s always good to test each of the exported procedures to ensure that they work as expected. The protocol module currently exports only one procedure—the parseMessage procedure—so you only need to write tests for it.

There are multiple ways to test code in Nim, but the simplest is to use the doAssert procedure, which is defined in the system module. It’s similar to the assert proce-dure: it takes one argument of type boolean and raises an AssertionFailed excep-tion if the value of that boolean is false. It differs from assert in one simple way:

assert statements are optimized out when you compile your application in release mode (via the -d:release flag), whereas doAssert statements are not.

RELEASE MODE By default, the Nim compiler compiles your application in debug mode. In this mode, your application runs a bit slower but performs checks that give you more information about bugs that you may have acciden-tally introduced into your program. When deploying your application, you should compile it with the -d:release flag, which puts it in release mode and provides optimal performance.

Let’s define an input and then use doAssert to test parseMessage’s output.

when isMainModule:

block:

Listing 3.11 Testing your new functionality The magical result variable

You may be wondering where the result variable comes from in listing 3.10. The answer is that Nim implicitly defines it for you. This result variable is defined in all procedures that are defined with a return type:

proc count10(): int = for i in 0 .. <10:

result.inc

assert count10() == 10

This means that you don’t need to repeatedly define a result variable, nor do you need to return it. The result variable is automatically returned for you. Take a look back at section 2.2.3 for more info.

The < operator subtracts 1 from its input, so it returns 9 here.

The when statement is a compile-time if statement that only includes the code under it if its condition is true. The isMainModule constant is true when the current module hasn’t been imported.

The result is that the test code is hidden if this module is imported. Begins a new scope (useful for isolating your tests)

let data = """{"username": "John", "message": "Hi!"}"""

let parsed = parseMessage(data) doAssert parsed.username == "John"

doAssert parsed.message == "Hi!"

Add the code in listing 3.11 to the bottom of your file, and then compile and run your code. Your program should execute successfully with no output.

This is all well and good, but it would be nice to get some sort of message letting you know that the tests succeeded, so you can just add echo("All tests passed!") to the bottom of the when isMainModule block. Your program should now output that message as long as all the tests pass.

Try changing one of the asserts to check for a different output, and observe what happens. For example, removing the exclamation mark from the doAssert parsed.message == "Hi!" statement will result in the following error:

Traceback (most recent call last) protocol.nim(17) protocol

system.nim(3335) raiseAssert system.nim(2531) sysFatal

Error: unhandled exception: parsed.message == "Hi" [AssertionError]

If you modify the protocol module and break your test, you may find that suddenly you’ll get such an error.

You now have a test for the correct input, but what about incorrect input? Create another test to see what happens when the input is incorrect:

block:

let data = """foobar"""

let parsed = parseMessage(data)

Compile and run protocol.nim, and you should get the following output:

Traceback (most recent call last) protocol.nim(21) protocol_progress protocol.nim(8) parseMessage json.nim(1086) parseJson json.nim(1082) parseJson json.nim(1072) parseJson json.nim(561) raiseParseErr

Error: unhandled exception: input(1, 5) Error: { expected [JsonParsingError]

Uses the triple-quoted string literal syntax to define the data to be parsed. The triple-quoted string literal means that the single quote in the JSON doesn’t need to be escaped.

Calls the parseMessage procedure on the data defined previously

Checks that the username that parseMessage parsed is correct Checks that the message that parseMessage parsed is correct

An exception is raised by parseJson because the specified data isn’t valid JSON. But this is what should happen, so define that in the test by catching the exception and making sure that an exception has been raised.

block:

An ideal way for the parseMessage proc to report errors would be by raising a custom exception. But this is beyond the scope of this chapter. I encourage you to come back and implement it once you’ve learned how to do so. For now, let’s move on to generat-ing JSON.

In document Nim in Action (Page 92-98)