We have also included the scripts we used to parse the output from each of the Lox tests in the `Parsing-results` directory. For convenience, copy all of the output log files to the `Parsing-results` directory and run `./parse_data.sh`. This is a python script that uses Python 3.8+ and depends on `numpy`, `matplotlib`, and `pandas` which can be installed with `pip3`.
To run all tests in fast mode, output the results to the `Parsing-results` directory, and generate the table (`performance_stats.csv`, our Table 4) and graphs (`Blockage-response-time.pdf` and `core-users.pdf`, our Figures 1 and 2) used in our paper, run:
This takes 5–6 hours to complete on the 2011-era E7-8870 processor we used for the measurements in the paper. Newer processors will likely take less time. Note that with the `fast` feature, it is possible that some tests may only yield 0 or 1 data points, in which case you will see `0` for the times and/or stddevs in the resulting `performance_stats.csv` table for those tests.
Note that our implementation is coded such that the reachability certificate expires at 00:00 UTC. A workaround has been included in each test to pause if it is too close to this time so the request won't fail. In reality, if the bucket is still reachable, a user could simply request a new reachability token if their request fails for this reason (a new certificate should be available prior to the outdated certificate expiring).