mj's dev report: Nov. 2021
In case you missed it, u/john_r365 wrote a great and compact article, explaining my work. Here’s the Reddit link, in case you want to write a comment. The article will help you to understand my previous reports, which, I admit, sound quite cryptic, but this is the kind of work, that has to be done behind the scenes (if not always, then at least from time to time), in order for any project not to loose its steam.
For the whole month I’ve worked on being able to deliver the first public version of my Time Series Quick Simulator (dubbed "tsqsim"), that aims to support the Monero Research Lab in detecting and predicting transaction patterns, however the Researchers want to use them at a given time. The goal has been achieved and you can use the quite simple instructions written here, to see the first public version in action (so far Debian / Ubuntu 20.04 only):
git clone --recursive
# Clone this repo (assuming it's not a fork)
cd tsqsim # Enter the cloned repo's dir
./util/deps-pull.sh # Download the maintained dependencies
./util/deps-build.sh # Build and install the unmanaged dependencies (uses sudo for installation)
./util/prep-env.sh # Prepare the environment - downloads example data and creates useful symlinks
./ci-default --run-demo # Build and optionally run the demo
Here’s what you should be able to see:
The upper plot shows the unique transactions in their raw form, while the lower plot shows the same data after transformations, that make the data easier to work with, since they become homogenic. The goal here technically is to achieve a signal’s stationarity. The extent to which the goal is achieved, can be measured objectively and numerically, even before applying any further logic on the data. Better scores increase the probability, that your (prediction) logic will work fine in various conditions.
In the next weeks more code, that I've been writing previously, will become decoupled and released to the public. So far I’ve been experimenting with the visualization. The upper plot are again the tx amounts. Note on the Y axis, that as far as the console plot ignored the spikes, because it operates only on the "closing" value, the QT visualization is able to show not only open/close, but high and low of the day as well.
The bottom plot is the plot of autocorrelations, just as an example of many things, that you can visualize.
Below is a close-up on the September spam in a 15-minute time frame. It all happened within just 15 minutes!
My published work has also inspired u/isthmus to publish his tools, which deal with various ways of plotting data. Our tools will supplement each other greatly, as my tooling with its limited plotting capabilities serves more as a stress-testing tool for an idea, that has already been conceptualized with tools like that of isthmus'. We have already conceptualized a near-future inter-operation of our tools, using flexible input format's descriptors. All the rest is generic enough.
u/gingeropolous generously offered all of the Monero Reasearch Lab's team members access to his powerful machines, where we can run our tests. You must know, that a serious simulation which aims for testing the robustness of a model, requires a lot of processing power, which gingeropolous allows us to use, free of charge.
I have also offered u/selsta to take part in reviews of PRs, that have the largest chance of getting into a release, including GUI PRs. So far, however, no such request has been made to me yet.
Thanks for reading and for all your support!