Adventures in algorithmic trading on the Runescape Grand Exchange

Adventures in algorithmic trading on the Runescape Grand Exchange(tristanrhodes.com)

186 points by partypete 1 year ago | 44 comments

soared 1 year ago |

I did this same thing but via a google sheet, really fun either how excellent the wiki’s API is. The profit per hour shown here is abysmal - you’d make more money doing pretty much anything.

I’d like to see market-level and category-level analysis on price trends (ie all magic-combat gear is trending up, or this specific item has a lagging relationship with another).

If anyone is curious, prices may be out of date but here is my profitability spreadsheet, time inputs, etc for various things using the api. This is better than the wiki since it can take into account trade limits, compare unlisted methods, hourly upkeep costs, etc. https://docs.google.com/spreadsheets/u/0/d/1iekEqInA2PwEuj1W...

You can tell likely from the methods I’m botting - turns out writing bots is even more fun than actually playing. Too bad private servers are being taken down through legal means, as I can bot there without ethical concerns.

partypete 1 year ago | |

Keep in mind that the actual profit if the bot were not randomly choosing between each model version would be about 600k/hour. You're conflating the experimental profit, wherein each model only has access to 1/3 of the potential trades, to what the profit would be if only the best model were making trades.

Also these bots have been running for over a year almost every day without being banned.

Salgat 1 year ago | |

Wait why is botting okay if it's on a private server? You mean the other players are okay with it?

nojs 1 year ago |

Haha, in the heyday of Runescape it used to be possible to simply have buy and sell orders sitting in the GE for the same rare items simultaneously, and over time they’d both get filled for a profit. I guess you would call it “market making”.

sharperguy 1 year ago | |

I used to wake up before school, dump my entire stack of gold on the GE for bones (just the regular ones) at a few gp below average price. By the end of the day the order would usually be fulfilled. Then I'd dump them back on the market at a few gp above average price and it would be fulfilled by the morning.

Most days it would net me a small profit, ocasionally prices would fall slightly throughout the day and I'd lose a little, but equally some days it would rise and I'd make extra.

Spivak 1 year ago | | |

How could you lose money on a standing sell order for a higher price than what you bought it for?

andrepd 1 year ago | | |

You needed an FPGA to pull your orders on underlying price moves /s

mhh__ 1 year ago | |

This is indeed how market making works.

bboygravity 1 year ago | |

This is actually kind of how the US market works with essentially 1 market maker who is by far the largest and who also happens to be the largest political donor (100 million a year to gop alone). Oh, and like Bernie Madoff, the owner of that market maker also owns a hedge fund (which like Bernie's makes completely impossible profits year after year). The hedge fund is of course a totally seperately run business from the market maker /s /s /s

So that actually sounds quite realistic to me from the market maker point of view of how things actually work in the US.

postexitus 1 year ago | | |

This is actually how markets are made - do you think a second hand book shop operates any differently?

next_xibalba 1 year ago | | |

You appear to be referencing Jeff Yass and Susquehanna International Group. Why are you being cagey about saying so? Their hedge fund has had losing years, whereas, as far as I recall, Madoff never reported a loss. On what information are you basing your claims?

kayson 1 year ago |

This reminds me of when World of Warcraft had a mobile app that allowed you to both browse the auction house and list or buy items. The API was pretty trivial to reverse engineer, making it easy to write an auction house bot. I never got as sophisticated as any ML algorithm, just snagged grossly mispriced items and bought/sold based on the last week's price range; it was mostly about volume. Made a tidy profit, bought my 1mil gold mount then gave up. The app was taken down soon after, no doubt in part because I'm sure I was not the only person doing this.

bongodongobob 1 year ago | |

WoW never cared about playing the market, it was probably taken down due to the way it was implemented. To be an external app, it probably had accounts idling and refreshing the AH and pushing that out to the app. I used the Auctioneer in game addon for a decade.

alex1138 1 year ago |

I'm not sure you can say enough good things about RS in general, aside from the very legitimate complaints the community has about the ownership, but basically it's a 20+ year game that's still getting updates, still getting new weapons and quests and level caps

knuckleheads 1 year ago |

I did something like this on Hypixel for a month or two. Watched the prices for inputs to various crafting recipes as well as the craft recipes output prices themselves on a loop, and whenever it hit a certain profit level, play a chime and I would go run some crafts. Easy enough to pull the info off of https://wiki.hypixel.net , and https://wiki.hypixel.net/Travel_Scrolls were the ones that most often had good returns. You needed a certain skill/level to make them, so the prices didn't even out too fast. Got tired of it eventually as the game wasn't so deep, but it's fun to mess around with for sure.

mhh__ 1 year ago | |

Arbitrage much more fun when r=0

orange_puff 1 year ago |

I really enjoyed the article, as I am also a nerd who has played runescape for what seems like forever now, and now most of my interaction with the game is via programming. I have two types of bots I use mostly, color bots, where a screenshot is taken and objects are detected by their surrounding pixel colors, and basic click bots. I've actually found that ~100 lines of python code using pyautogui is more than enough to automate tons of annoying aspects of the game.

I am curious, is your Java client one of the many open source bot clients that actually calls into client code? Or is it some type of click script which does some repetitive inputs?

I have had bad luck with the former in terms of getting banned.

beoberha 1 year ago |

Sorry if I’m just bad at reading comprehension, but what exactly is being modeled here? It sounds like the author is just getting price data not unlike a candlestick chart. What I don’t understand is if the trading is happening within the 5 minute windows he mentions or if model is finding patterns across timeframes?

Frummy 1 year ago | |

So I don’t know this stuff so I’ll stochastically parrot a reply here of my understanding of what I just read.

The ML model is trained on the price spread and volume data collected with the cronjob every 5 minutes and stored in a database, and the trades with their respective data, but on previously collected data from 63 days up to 14 days earlier and then validated on the remaining later 14 days of data, apparently to not have temporal leakage which I googled and means unintentional backflow of training data which screws up the prediction. So to answer your question the model finds patterns over multiple weeks of data, to rank the best trading choices in the present moment. Then the java component executes the trade in-game with the player character.

What I personally don’t understand is if this data collection and training is rolling continually, or if he let it be trained one shot and then it’s smart enough to remain profitable. Don’t entirely understand the 49 day training, 14 day validation thing either.

partypete 1 year ago | | |

That’s right - I retrained the model every 7 days. These experiment results are from the most recent one-week period.

partypete 1 year ago | |

The training data is composed of the bot’s previously executed trades from the last 63 days joined on the price/volume data. So the model discovers patterns in the price/volume data that have historically lead to well-performing trades. The dependent variable is gold/second generated from a historical trade.

beoberha 1 year ago | | |

Thanks! But how was the bot running before the training data existed? I think that’s what I was missing. The baseline?

naming_the_user 1 year ago |

I made approx 1K GBP as a young teenager trading in RuneScape prior to the introduction of the GE.

Haggle people down to say 6 mil for a party hat I'd sell for 6.5, repeat, when I quit the game I sold it all on eBay, my mum was rather surprised when the money came in.

Hourly rate? Awful.

I very much doubt that I'd be where I am today without that experience though.

dleink 1 year ago | |

What line of work are you in now?

naming_the_user 1 year ago | | |

Entrepreneur, multiple income streams, software is one of them.

rightbyte 1 year ago | |

> for a party hat

O dear I've forgot about how saught after those were.

naming_the_user 1 year ago | | |

One of my regrets is that I never quite got to own a Blue.

mrklol 1 year ago |

"two cronjobs that interact with the OSRS Wiki API: one that polls it every 5 minutes, and another that polls it every hour"

Why two? Isn’t the latter useless as you have the same information in the current 5m interval?

noitpmeder 1 year ago |

Has anyone done similar experiments with EVE's marketplace?

night862 1 year ago | |

Yes, a LOT of people.

There is a lot of this in EvE Online and it's a valid and welcome play style. Unfortunately, everybody will tell you its opsec if you ask for details.

If you want more information about EvE Online's extremely deep and intricate economy (Essentially everything is built by players who mine ore to make components who resell to integrators that have associated distributors and clients and so forth) you can check Oz on youtube who makes market and market analysis content ( https://www.youtube.com/@OZeve )

Very closely related, EvE Online now has an official MS Excel plugin from Microsoft with full API integration with EvE Online. This game is an effective way to become an expert using MS Excel and its full, powerful integrations: Oz' Quick Guide: https://youtu.be/eAogNNeDZuo

EvE Online also has a convention where our EvE Ted Talks are presented to one another (like defcon) called Fanfest. Here's 51 minutes: Fanfest 2023 "Market Spreadsheet Masterclass" https://www.youtube.com/watch?v=6Ql6IU1eZpM

If you have an idea, or want to see if you can do this, the free-to-play account can for zero dollars if you already have an MS Excel license or are able to integrate your own solution. The limitation is you will start with zero capital, knowledge, intuition, and F2P accounts can't make nearly as many orders as a fully skilled character.