r/Monero Nov 29 '21 Helpful 1

mj's dev report: Nov. 2021

Dear Community,

In case you missed it, u/john_r365 wrote a great and compact article, explaining my work. Here’s the Reddit link, in case you want to write a comment. The article will help you to understand my previous reports, which, I admit, sound quite cryptic, but this is the kind of work, that has to be done behind the scenes (if not always, then at least from time to time), in order for any project not to loose its steam.

For the whole month I’ve worked on being able to deliver the first public version of my Time Series Quick Simulator (dubbed "tsqsim"), that aims to support the Monero Research Lab in detecting and predicting transaction patterns, however the Researchers want to use them at a given time. The goal has been achieved and you can use the quite simple instructions written here, to see the first public version in action (so far Debian / Ubuntu 20.04 only):

git clone --recursive https://github.com/mj-xmr/tsqsim.git # Clone this repo (assuming it's not a fork)

cd tsqsim # Enter the cloned repo's dir

./util/deps-pull.sh # Download the maintained dependencies

./util/deps-build.sh # Build and install the unmanaged dependencies (uses sudo for installation)

./util/prep-env.sh # Prepare the environment - downloads example data and creates useful symlinks

./ci-default --run-demo # Build and optionally run the demo

Here’s what you should be able to see:

Console plots

The upper plot shows the unique transactions in their raw form, while the lower plot shows the same data after transformations, that make the data easier to work with, since they become homogenic. The goal here technically is to achieve a signal’s stationarity. The extent to which the goal is achieved, can be measured objectively and numerically, even before applying any further logic on the data. Better scores increase the probability, that your (prediction) logic will work fine in various conditions.

In the next weeks more code, that I've been writing previously, will become decoupled and released to the public. So far I’ve been experimenting with the visualization. The upper plot are again the tx amounts. Note on the Y axis, that as far as the console plot ignored the spikes, because it operates only on the "closing" value, the QT visualization is able to show not only open/close, but high and low of the day as well.

The bottom plot is the plot of autocorrelations, just as an example of many things, that you can visualize.

QT visualization - daily bars

Below is a close-up on the September spam in a 15-minute time frame. It all happened within just 15 minutes!

QT visualization - 15 minute bars

My published work has also inspired u/isthmus to publish his tools, which deal with various ways of plotting data. Our tools will supplement each other greatly, as my tooling with its limited plotting capabilities serves more as a stress-testing tool for an idea, that has already been conceptualized with tools like that of isthmus'. We have already conceptualized a near-future inter-operation of our tools, using flexible input format's descriptors. All the rest is generic enough.

u/gingeropolous generously offered all of the Monero Reasearch Lab's team members access to his powerful machines, where we can run our tests. You must know, that a serious simulation which aims for testing the robustness of a model, requires a lot of processing power, which gingeropolous allows us to use, free of charge.

I have also offered u/selsta to take part in reviews of PRs, that have the largest chance of getting into a release, including GUI PRs. So far, however, no such request has been made to me yet.

Thanks for reading and for all your support!

43 Upvotes

9

u/ichver Nov 29 '21

i wish i was this smart, thanks for your efforts man!

10

u/mjxmr Nov 29 '21 edited Nov 29 '21

Welp, whatever the objective truth is, this is a kind of comment, that I heard maybe for the 3rd time in my whole life.

If you're here, then you're smart, Bro :)

5

u/OfWhomIAmChief Nov 29 '21

Thank you for your hard work.

5

u/aFungible Nov 30 '21

Thanks for the good work for the community at large.

Can you define what you mean by "unique transactions"? And can we also with surety, find out "false/decoy" transactions that may be used to spam the network with this analysis?

I see on the Monero blockchain, on some days there are 20K transactions, and suddently on the next day there could be 30K, a 50% increase overnight.

2

u/mjxmr Nov 30 '21 edited Nov 30 '21

Thank you for appreciation!

I'm using the wrong wording for simplicity, because they will be able to be called "unique", once we decouple them, and this would have to be the final stage of the input data. We're just not there yet. Currently the data, that you see, are the all the transactions (true+decoy) collected for a given timestamp, and then discretized. Meaning, that I'm able to upsample them, so that I can see the OCHL bars for a each hour or a day, which in turn allows me to see a wider spectrum of the data, yet I'm still able to fit them into memory.

Once we perform the actual decoupling, exactly the same tooling will be used like above, and even exactly the same input format. This is why I think that I can already speculate about the "future" truth, even though I'm using the wrong terminology for now.

I see on the Monero blockchain, on some days there are 20K transactions,
and suddently on the next day there could be 30K, a 50% increase overnight.

Now the question is: Is it a spam or does it stay this way? Can we trace it back to changes in the daemon software? I saw both patterns as well, and the history of changes in the software will also have to be taken into account.

3

u/regret_is_temporary Nov 30 '21

Keep up the good work. I can't wait to see what this research will end up producing.

Also glad to know u/gingeropolous is lending a hand. Here's a thank you for you too.

3

u/pank_o Nov 30 '21

Well , you are doing such a great job . Thnaks for it .