PureBasic - MsgPack - Performances
Performance benchmarks for the PB-MessagePack include.
Please refer to the Performances section for the warnings and conclusions.
Open questions raised by these results are tracked in Investigations.md.
Workloads
Each folder has a RunPerfTests.ps1 script that runs its benchmarks:
MassiveReadAmount/- 1M small fields written, read and skipped, with explicit and implicit typing.StringsAndBinary/- Strings of 8 B to 64 KB, and binaries of 1 KB and 1 MB, written and read.FlatMaps/- 100K maps of 16 entries written and read.DeepNesting/- Arrays nested 1 to 128 levels deep, built, examined and measured. (PureBasic only)NullChecks/- Small fields written and read with and without#MsgPack_DisableNullChecks. (PureBasic only)
The first three are compared to msgpack (Python) and MessagePack-CSharp (.NET).
The scripts prompt for the PureBasic version, architecture and backend.
The results are saved in Results/, as one CSV per folder, PureBasic version, architecture and backend.
Results
Hardware:
- Proxmox VM running Windows 10
- 10 Intel Core i5-13500 cores (No E/P core pinning)
- PureBasic 6.41
- Python 3.14 with msgpack 1.2.3
- .NET 10 with MessagePack-CSharp 3.1.12
Times are in nanoseconds per field, per map or per nested structure, lower is better.
Every implementation produced the same bytes and read back the same values.
PureBasic 6.41 is used everywhere, except in the last table.
Compared to other implementations
x64 with the C backend, the fastest of each row in bold.
| Workload | Op | PureBasic | Python | .NET |
|---|---|---|---|---|
| 1M small fields 1 | Write | 11.9 | 50.9 | 5.4 |
| Read | 12.7 | 52.4 | 10.6 | |
| Skip | 18.3 | 40.2 | 29.4 | |
| Strings, 8 B | Write | 59.6 | 62.0 | 13.2 |
| Read | 46.6 | 73.4 | 16.2 | |
| Strings, 1 KB | Write | 720.9 | 556.0 | 804.8 |
| Read | 570.0 | 341.3 | 150.0 | |
| Strings, 64 KB | Write | 45,917 | 33,932 | 39,848 |
| Read | 31,473 | 16,390 | 23,045 | |
| Binary, 1 KB | Write | 190.6 | 556.6 | 576.1 |
| Read | 50.2 | 311.3 | 204.0 | |
| Binary, 1 MB | Write | 207,744 | 574,825 | 438,025 |
| Read | 54,812 | 465,075 | 59,459 | |
| Maps of 16 entries | Write | 1,793 | 927.2 | 209.2 |
| Read | 1,858 | 1,962 | 292.1 |
1 1M fields of mixed integers, floats and short strings, with implicit typing. PureBasic writes into a preallocated buffer.
Architectures & backends
PureBasic only.
| Scenario | x64 C | x64 ASM | x86 C |
|---|---|---|---|
| Small fields, explicit write | 11.0 | 18.0 | 13.9 |
| Small fields, explicit read | 9.8 | 15.9 | 8.8 |
| Small fields, implicit write | 11.9 | 25.4 | 14.2 |
| Small fields, implicit read | 12.7 | 25.3 | 11.5 |
| Small fields, skip | 18.3 | 57.9 | 23.2 |
| Small fields, write, 64 KB growth 2 | 37.9 | 51.5 | 44.7 |
| Small fields, write, exact growth 2 | 441.0 | 461.6 | 503.3 |
| Small fields, read, null checks off 3 | 11.4 | 19.5 | 11.4 |
| Small fields, read, null checks on 3 | 12.2 | 20.2 | 11.4 |
| Strings, 1 KB UTF-16, write | 526.0 | 539.1 | 599.0 |
| Strings, 1 KB UTF-16, read | 282.7 | 298.8 | 289.1 |
| Maps of 16 entries, write | 1,793 | 3,472 | 2,189 |
| Maps of 16 entries, read | 1,858 | 3,232 | 1,782 |
2 The buffer starts at 64 bytes and grows in multiples of 64 KB, or by the exact size of each write. (GrowthIncrement of 1)
3 With and without #MsgPack_DisableNullChecks.
Deep nesting
x64 with the C backend, arrays of 3 integers nested N levels deep.
| Depth | Build | Examine | Examine, unsafe | MsgPackGetFieldFullSize |
|---|---|---|---|---|
| 1 | 283.3 | 177.8 | 134.9 | 80.4 |
| 2 | 453.4 | 363.2 | 261.4 | 115.8 |
| 4 | 1,002 | 825.2 | 464.1 | 177.3 |
| 8 | 2,226 | 2,154 | 906.9 | 308.9 |
| 16 | 6,414 | 6,155 | 1,807 | 546.5 |
| 32 | 21,888 | 21,134 | 4,367 | 985.9 |
| 64 | 76,370 | 72,715 | 7,774 | 1,997 |
| 128 | 278,388 | 269,656 | 15,803 | 3,812 |
PureBasic 6.21 vs 6.41
x64 with the C backend, the new string handling of 6.40 being the main difference.
| Scenario | 6.21 | 6.41 | Speedup |
|---|---|---|---|
| Small fields, implicit write 4 | 23.0 | 11.9 | 1.9x |
| Small fields, implicit read 4 | 26.4 | 12.7 | 2.1x |
| Small fields, skip | 14.7 | 18.3 | 0.8x |
| Strings, 8 B, write | 162.8 | 59.6 | 2.7x |
| Strings, 8 B, read | 164.5 | 46.6 | 3.5x |
| Strings, 1 KB, write | 1,971 | 720.9 | 2.7x |
| Strings, 1 KB, read | 1,412 | 570.0 | 2.5x |
| Strings, 64 KB, write | 113,872 | 45,917 | 2.5x |
| Strings, 64 KB, read | 77,056 | 31,473 | 2.4x |
| Strings, 1 KB UTF-16, write | 2,026 | 526.0 | 3.9x |
| Strings, 1 KB UTF-16, read | 1,065 | 282.7 | 3.8x |
| Binary, 1 KB, write | 173.4 | 190.6 | 0.9x |
| Binary, 1 KB, read | 41.2 | 50.2 | 0.8x |
| Maps of 16 entries, write 4 | 3,221 | 1,793 | 1.8x |
| Maps of 16 entries, read 4 | 3,478 | 1,858 | 1.9x |
| Nested 128 levels, examine | 263,158 | 269,656 | 1.0x |
4 Includes strings, one per record of 8 fields, and one per map key.
The x64 ASM and x86 C results show the same 2 to 3 times speedup on strings.