Why Android developers should consider FlatBuffers over JSON


FlatBuffers vs JSON

You may be asking — why use new tools if we already have standard JSON, JSON-parser like GSON, and Jackson.

Well, give FlatBuffers a try and you’ll find that they’re significantly faster than JSON.

What are FlatBuffers?

FlatBuffers are an efficient cross platform serialization library for C++, C#, C, Go, Java, JavaScript, PHP, and Python. They were originally created at Google for game development, and other performance-critical applications.

Why use FlatBuffers?

  • Access to serialized data without parsing/unpacking — What sets FlatBuffers apart is that they represent hierarchical data in a flat binary buffer, in such a way that they can still be accessed directly without parsing and unpacking, while also still supporting data structure evolution (forwards/backwards compatibility).
  • Memory efficiency and speed — The only memory needed to access your data is that of the buffer. It requires zero additional allocations (at least in C++. Other languages may vary). FlatBuffers are also suitable for use with mmap (or streaming), requiring only part of the buffer to be in memory. Access is close to the speed of raw struct access with only one extra indirection (a kind of vtable) to allow for format evolution and optional fields. FlatBuffers are aimed at projects where spending time and space (many memory allocations) to be able to access or construct serialized data is undesirable, such as in games, or any other performance sensitive applications. See the benchmarksfor details.
  • Flexible — Having optional fields mean that not only do you get great forwards and backwards compatibility (increasingly important for long-lived games: you don’t have to update all data with each new version). It also means you have a lot of choice in what data you write and what data you don’t, and how you design data structures.
  • Tiny code footprint — FlatBuffers require only small amounts of generated code, and just a single small header as the minimum dependency, which is very easy to integrate. Again, see the benchmark section for details.
  • Strongly typed — Errors happen at compile time rather than manually having to write repetitive and error prone run-time checks. Useful code can be generated for you.
  • Convenient to use — Generated C++ code allows for terse access and construction code. Then there’s the optional functionality for parsing schemas, and JSON-like text representations at runtime run efficiently if you need them (faster and more memory efficient than other JSON parsers).
  • Cross platform code with no dependencies — C++ code will work with any recent gcc/clang and VS2010. Comes with build files for the tests & samples (Android .mk files, and cmake for all other platforms).

Who uses FlatBuffers?

  • I tried FlatBuffers and got the significant improvement in app performance.
  • Cocos2d-x, the #1 open source mobile game engine, uses them to serialize all their game data.
  • Facebook uses them for client-server communication in their Android app. They have a nice article explaining how it speeds up loading their posts.
  • Fun Propulsion Labs at Google uses them extensively in all their libraries and games.

How significant are the app performance increases?

  • Parsing speed. It took 35 ms to parse a JSON stream of 20 KB (a typical response size), which exceeds the UI frame refresh interval of 16.6 ms. With this approach, we were unable to load data on demand from our disk cache without observing frame drops (visual stutters) while scrolling.
  • Parser initialization. A JSON parser needs to build field mappings before it can start parsing, which can take 100 ms to 200 ms, substantially slowing down the application start time.
  • Garbage collection. A lot of small objects are created during JSON parsing, and in our explorations, around 100 KB of transient memory was allocated when parsing a JSON stream of 20 KB, which placed significant pressure on Java’s garbage collector.

FlatBuffer vs JSON

I have tried to parse 4 mb of JSON file with both FlatBuffer and JSON.

FlatBuffer tooks 1–5 ms and JSON took around 2000ms.And no GC was called in android app during FlatBuffer use.But GC was called so many time while using JSON. So, UI was freezing in case of JSON(only option left with JSON is to parse it in background thread).

E50b6 1lonwaexndshfritaqan46q

How to use FlatBuffer?

Here is my github sample project (with example)which will simply explain step by step how to use FlatBuffer.

# FlatBuffer Android Sample Application
This app shows how fast flat buffer works when we compare it with json.
### How to start with flatBuffer

$ git clone https://github.com/google/flatbuffers.git

$ cd flatbuffers

* Run the command on the basis of your platform

$ cmake -G "Unix Makefiles"
$ cmake -G "Visual Studio 10"
$ cmake -G "Xcode"

* now build for your platform as usual. This should result in a flatc executable
* Now create your schema file with extension .fbs. Guide to write a schema can be found [here]("https://google.github.io/flatbuffers/flatbuffers_guide_writing_schema.html").And also have your sample json file.

$ ./flatc -j -b schema.fbs sample.json

* This will create few java file and one bin file. Java files are like model(POJO for flatBuffer) of your json.Place the java files in your application package and bin file in raw folder(bin file is only for testing as it is converted to byte that is to be passed to flatbuffer for testing).
* Now we have to get flatbuffer jar file.

$ cd flatbuffers
$ cd java
$ mvn install
This will download all the dependencies.
$ cd target

* Here you will get the flatbuffers-java-1.3.0-SNAPSHOT.jar file that you have to put it in your libs folder of android project.
* For rest you can see my [sample project](https://github.com/amitshekhariitbhu/FlatBuffer).

## Major steps:
* Prepare your schema.fbs.
* Have a sample json.
* Build flatBuffer google project to generate your java files to be used in main application.
* Generate java files.

That’s it for now.

Reference: Google’s FlatBuffers website and Facebook’s blog on FlatBuffers.

Amit Shekhar

Also, Let’s become friends on Twitter, Linkedin, Github, Quora, and Facebook.