confused of paresr callback's behavior of null object: remove or discarded?

[gino0717]

hi sir,

I'm studying the document of Parser Callbacks

I replace some of the values to null of the example , then expect the parser would remove the object if the value is null.

the input goes like :

{
        "Image": {
            "Width":  null,
            "Height": 600,
            "Title":  "View from 15th Floor",
            "Thumbnail": {
                "Url":    "http://www.example.com/image/481989943",
                "Height": 125,
                "Width":  null
            },
            "Animated" : false,
            "IDs": [116, 943, 234, 38793]
        }
}

Both "Width" under "Image" and "Thumbnail" are null.

in version 2.1.1 which directly installed from apt-get gives the result just as I expected, both the "Width" disappear due to the null value:

{
    "Image": {
        "Animated": false,
        "Height": 600,
        "IDs": [
            116,
            943,
            234,
            38793
        ],
        "Thumbnail": {
            "Height": 125,
            "Url": "http://www.example.com/image/481989943"
        },
        "Title": "View from 15th Floor"
    }
}

but in version 3.10.5 it gives the result like:

{
    "Image": {
        "Animated": false,
        "Height": 600,
        "IDs": [
            116,
            943,
            234,
            38793
        ],
        "Thumbnail": {
            "Height": 125,
            "Url": "http://www.example.com/image/481989943",
            "Width": <discarded>
        },
        "Title": "View from 15th Floor"
    }
}

the second "Width" in Thumbnail becomes "discarded" , and the "discarded" itself cannot be parsed if I feed the filtered json back to parser by :
json parse_again=json::parse(j_filtered.dump());

it gives

terminate called after throwing an instance of 'nlohmann::detail::parse_error'
what(): [json.exception.parse_error.101] parse error at line 1, column 148: syntax error while parsing value - invalid literal; last read: '"Width":<'

When does the parser give the null object a "discarded" value or just remove the object if filtered by the parser callback? Could I chose which behavior it acts?

here's my original code:

#include <iostream>
#include <iomanip>
#include <nlohmann/json.hpp>

using json = nlohmann::json;

int main()
{
    // a JSON text
    auto text = R"(
    {
        "Image": {
            "Width":  null,
            "Height": 600,
            "Title":  "View from 15th Floor",
            "Thumbnail": {
                "Url":    "http://www.example.com/image/481989943",
                "Height": 125,
                "Width":  null
            },
            "Animated" : false,
            "IDs": [116, 943, 234, 38793]
        }
    }
    )";

    // parse and serialize JSON
    json j_complete = json::parse(text);
    std::cout << std::setw(4) << j_complete << "\n\n";

    // define parser callback
    json::parser_callback_t cb = [](int depth, json::parse_event_t event, json & parsed)
    {
        // skip object elements with key "Thumbnail"
       // if (event == json::parse_event_t::key and parsed == json("Thumbnail"))
         if (event == nlohmann::json::parse_event_t::value and parsed.is_null())
        {
            return false;
        }
        else
        {
            return true;
        }
    };

    // parse (with callback) and serialize JSON
    json j_filtered = json::parse(text, cb);
    std::cout << std::setw(4) << j_filtered << '\n';
    
    json parse_again=json::parse(j_filtered.dump());
}

[dingminghbut]

<discarded> is an internal JSON value used by the parser callback mechanism. It is not valid JSON syntax, so a dumped value containing it should not be fed back into json::parse().

If the goal is "remove all object members whose value is null", I would not rely on filtering the value event directly. It is easier and more predictable to parse normally, then erase null members recursively:

#include <nlohmann/json.hpp>

using json = nlohmann::json;

void erase_null_object_members(json& j)
{
    if (j.is_object()) {
        for (auto it = j.begin(); it != j.end(); ) {
            if (it->is_null()) {
                it = j.erase(it);
            } else {
                erase_null_object_members(*it);
                ++it;
            }
        }
    } else if (j.is_array()) {
        for (auto& element : j) {
            erase_null_object_members(element);
        }
    }
}

int main()
{
    json j = json::parse(text);
    erase_null_object_members(j);
    std::cout << j.dump(4) << '\n';
}

This produces a normal JSON value that can safely be dumped and parsed again.

Parser callbacks are best when you know what structural element you want to skip while parsing. For example, if you want to remove a specific object member such as "Thumbnail", returning false from the key event is the right shape:

json::parser_callback_t cb = [](int, json::parse_event_t event, json& parsed) {
    if (event == json::parse_event_t::key && parsed == "Thumbnail") {
        return false; // skip this object member
    }
    return true;
};

But for "remove every null value anywhere", a post-parse recursive cleanup is simpler and avoids the ambiguity around where the discarded value sits during parsing.

Also, if you can, test with a current nlohmann/json release. Version 3.10.5 is old enough that callback edge cases may not match the current documentation exactly.