Reverse engineering the Path of Exile game protocol - Part 3: Introducing exileSniffer

Part 1: Obtaining the plaintext

Part 2: Decoding the protocol

Up to this point I haven't talked much about my workflow for extracting and examining the protocol data - so here a tool you can use to do it. This will walk through what it does and how it works.

Download here: https://github.com/ncatlin/exileSniffer

Requirements: Windows, winpcap  (If Wireshark works then this should work too)

Starting packet capture

Initial screen with no Path of Exile clients running

When exileSniffer starts it will start scanning the process list for Path of Exile clients and sniff on the default interface for logon packets. It doesn't matter which order the processes are started in, but it's essential that exileSniffer is launched and listening before login starts.

A game client is detected - exileSniffer is scanning its memory for keys

The player has logged in - we are waiting for keys and testing them against the logon traffic

We found the correct salsa key and have started decrypting the logon stream.

The above shows packet 0x4 from the login server highlighted - so far I've ignored it and it has no purpose disaplayed. If we look at the hex pane we can see a bunch of zeroes and a hex 12 at the end, which doesn't really tell us much. It comes after the authentication data so maybe it's a "Logon success" message. Try a bad password and report back if you care about the login process I guess.

Analysis pane showing a League List packet

For most packets I've encountered I've attempted to give them a meaningful description in the packets list, but for some with lots of interesting data I've added some analysis in the Analysis pane.

There are a few workflow features

• Filters and Filter Lists to prune out noisy packets and highlight interesting ones
• A hash search function in 'Utilities' so we can test if any data we find is a known hash
• Logging to disk - both raw and filtered streams
• A JSON pipe, which I'll talk about next

Using exileSniffer decryption with other tools

The deserialisation and decoding is done in entirely seperate threads to the analysis and display of the data. exileSniffer opens a named pipe so that external tools can subscribe to the same decoded data as the UI receives, which allows us to solve some problems in new ways.

Here are some example Python 3 scripts.

Tool #1: Monitoring player health

import json
import codecs

def connectToPipe():
    pipename = "\\\\.\\pipe\\ExilePipe"
    pipe = codecs.open(pipename, 'rb', encoding='utf-16-le')
    return pipe

if __name__ == "__main__":

    currentPlayerID = 0
    
    pipe = connectToPipe()
    while True:
        ln = pipe.readline()
        if len(ln) == 0: continue
        js = json.loads(ln)

        if js['MsgType'] == 'SRV_NOTIFY_PLAYERID':
            currentPlayerID = js['Payload']['ID1']
            continue

        if js['MsgType'] == 'SRV_MOBILE_UPDATE_HMS':
            if js['Payload']['Stat'] == 0: #life
                print(js['Payload']['NewValue'])
            continue
        
    pipe.close()

This should hopefully be straightforward.

• It looks for the SRV_NOTIFY_PLAYERID packet sent when joining the gameserver to figure out what our characters unique ID is.
• It looks for SRV_MOBILE_UPDATE_HMS packets sent whenever the health/mana/shield of a mobile changes.
• If the mobiles ID is our player, and the stat is Life, it prints the new value.

One improvement could be to read the players maximum health from the object creation packet and alert on low percentage.

All the packets I've found so far are listed in messageTypes.json in the application directory, but only the basic ones will be there. I suspect a huge amount of more obscure ones for certain skills, certain monsters, certain maps, crafts, vendor actions, league specific stuff, etc are all waiting to be found. You could also just enumerate them by injecting packets with the unknown ID's into the client, but if there is anything I don't need in life it's more of these packets to work through.

Tool #2: Monitoring party members

Here is a question someone asked on the PathOfExileDev subreddit.

Title: "Players joining your party not logged?"

Text: "It seems almost everything is in the log file, including changing instances, global chat, whispers, etc, but not members joining your party? Is this somewhere else in another log? Can I somehow keep track of when someone joins a party?"

The only reply:  "nope, the only log is clients.txt"

I feel like their expectations are already quite low if they consider the paltry contents of clients.txt to be 'almost everything' and seeing party member joins/quites doesn't seem like a big ask.

Looking into the party packets, it seems that the client doesn't get told 'X has joined, Y has left' but gets the whole party list every time the makeup of the party changes.

This Python script will track changes to the party and announce joiners and leavers.

import json
import codecs

def connectToPipe():
    pipename = "\\\\.\\pipe\\ExilePipe"
    pipe = codecs.open(pipename, 'rb', encoding='utf-16-le')
    return pipe

if __name__ == "__main__":
    inited = False
    oldNames = []
    
    pipe = connectToPipe()
    while True:
        ln = pipe.readline()
        if len(ln) == 0: continue
        js = json.loads(ln)

        if js['MsgType'] == 'SRV_PARTY_DETAILS':
            membersDict = js['Payload']['MemberList']
            updatedNames = [x['Name'] for x in membersDict if x['StatusText'] != 'Pending']
            
            if not inited:
                print("Party created with %d members"%(len(updatedNames)))
                inited = True
                oldNames = updatedNames
                continue

            newNames = list(set(updatedNames).difference(set(oldNames)))
            for x in newNames:
                print("%s joined the party"%x)
                
            leftNames = list(set(oldNames).difference(set(updatedNames)))
            for x in leftNames:
                print("%s left the party"%x)

            oldNames = updatedNames

        
    pipe.close()

Does this give an unfair advantage?

The first thing to consider is information: I was curious to see what the client was being sent that the player is not supposed to know about.

In general the server is quite good about only sending what the client needs to know - you can't see other players or monsters on the map unless they are almost on your screen. While you can see all of a players stats, devotions, XP, quests, etc - you can't see what items they are carrying or using - other than what is needed to render it.

I haven't played enough recently to remember if invisible things are a problem - but I suspect the 'is_hidden_monster' stat that some objects have could be made to lose it's effectiveness.

One thing I've noticed years ago when scanning game memory (and have found the cause for during this project) that probably could be exploited is the preload lists sent when you join a map. If a rogue exile, NPC grandmaster or strongbox exists: the server will tell the client as soon as it loads so the graphics can be preloaded. This allows you to load maps over and over again to look for something juicy without having to actually spend the time to explore them. My suggestion to the devs is to only send these to the client when the player is nearby, like when a new player joins the map and their totems are preloaded. The performance hit can't be that bad compared to people mass-loading new maps.

The second possible advantage is to be able to 'do' something that normal players can't do.

I know it's not going to be seen as much of a mitigation but it's worth highlighting that exileSniffer doesn't interact with the gameserver in anyway, the sole interaction with the client is reading key data while it is talking to the login server. This means players will still have to use other more intrusive programs to react to exileSniffer data. A google search for 'path of exile bots' shows that those programs are already out there and I'm not releasing anything that bot developers don't already know (and are using for a fee).

Further work

So I think the ability to write little scripts like that to interact with the games traffic is the coolest outcome from this project, but to write the above party script I had to go back and reverse the handling of the party info packet. This highlights the fact that huge swathes of the protocol are still left to be understood.

In particular:

• Items. This is the big one as it's what PoE is all about but with the classes, mods, corruptions, sockets, links and up to 6 items within items also needing decoding  - it's a lot of work. It does at least lookup the hash to the GGPK item ID.
• Objects. I've reversed most of the player creation packet so we can see handy things like all of the stats a player has and what microtransaction effects they are using, but monsters, npcs, pets, etc also need to be done.
• The ID triad. An object identifier is (DWORD, DWORD, WORD) but I've not found any particular use for the second and third items, other than occasionally referring to the object that caused something to happen to the target object.

It would also probably be possible to remove the requirement to start sniffing before logon by brute forcing the Salsa iteration value.

Reverse engineering the Path of Exile game protocol - Part 2: Decoding the protocol

This is effectively a puzzle, and it's really quite entertaining.

Part 1: Obtaining the plaintext

Part 3: Introducing exileSniffer (link fixed!)

Getting Started

After the loginserver hands us off to the gameserver we start getting a really noisy decrypted stream -  especially if you are in town with lots of people moving around - so it helps to move our character to an area of wilderness where not much is happening.

The packets we see are almost all prefixed with a 00 or a 01 and we see the same sets of 2 bytes quite regularly so (like with the login packets) we assume the first two bytes are packet types.

There are two ways of working out what a particular packet ID does:

Behavioural analysis - Empty Packets

The only time you see a certain packet ID sent to the server is when you perform a certain action.

A very trivial example is when we release the mouse button and see packet

00DC

Now we can catch 00DC packets in our decoder, mark it as uninteresting and filter it out..

Behavioural analysis - Constant Length Packets

Next step up is the packet we see when using potions (stored in one of 5 belt slots) of the format 0037XXXXXXXX. It is followed by 4 bytes (XX) at the end which clearly correspond to the potion slot we activated.

[0037]00000004 -> Potion slot 5
[0037]00000001 -> Potion slot 2

We add a check that the number always falls in the range of our available slots, then filter it out.

This technique can be extended to gain insight into the game mechanics, too.
Lets look at packet ID 0xDA, which appears when we drag the mouse around with the button held:

00DA000001E10000010B
00DA000001E60000010C
00DA000001E80000010C

This is clearly the two 4 byte coordinates of the mouse. Now we have a method of determining locations on a game map, so if we see a monster walking around and hover our mouse over them we can search for the coordinates in other packets to find their location bytes.

Additionally, since 00DA is the mouse being dragged and 00DC is the mouse being released - it doesn't take a lot of imagination to assume that other packets close to that range might also describe mouse activity.

Behavioural analysis - Variable Length Packets

This is trickier. PoE's Client/server comms are generally not self describing at all, so if you don't understand a logical packet then you have no idea where it ends and the next one begins. Starting is the hard part but after identifying more and more packet types you can usually pick them apart in a raw blob.

Does a certain packet contain variable length fields? Strings are an obvious one (always preceded by their length) but when a change in one byte results in a drastic change in the whole structure of the following packet then it's probably time to start reversing the binary.

Packet injection

Dynamic analysis is more fun when it's interactive, so to test a theory about what a packet field might be (or to just fiddle with it and see what happens) we want to be able to inject packets into the client at run-time and see what happens. I didn't want to do this when talking to the real game servers because -

• The client trusts the server. Bad input often makes it crash/hang in a variety of un-managed ways so if the server is even remotely as trusting of the client then it's might crash instance servers that people are playing on.
• Testing is noisy, especially when the data is malformed. The accounts are free but getting IP-banned would be a pain.
• On live servers there can be a lot of background activity which makes results hard to replicate.

The solution was to create a dummy server - just functional enough for a client (with modified crypt data) to log into as many times as we want with the same state each time. We can read everything the client sends and see what happens when we send it stuff.

"Do anything nice at the weekend Nia?"

Me inside: "I spent it reversing a video game to develop tortoise-choreography capability"

Me actually: "Nothing much, you?"

Binary analysis

There are hundreds of packet types and so many types of object, item, skill, monster, effect, etc that even if we could work each one out behaviourally - it's silly not to try and find handlers in the code and/or data files to help us. Ideally with so much data we want to automate the process too.

The joy of this process is using analysis of the data stream in conjunction with the binary and slowly building up a thorough understanding of both.

I couldn't see any obvious location for parsing the packet ID in binary ninja, so I fired up x64dbg to break in a location of code that I knew was triggered by incoming network data.

For this I plonked my character on a beach next to a monster called a Sand Spitter and let it... spit sand at me. Searching for relevant results in the module brought up a few dozen results - at least one of which was hit as a breakpoint:

Fortunately x64Dbg does handle unicode strings

After that it's simply a case of walking back through the call-stack to find the packet handling function that handles all incoming data.

From deserialisation to processing - a lazy shortcut

Reversing the deserialisation correctly is essential to being able to separate distinct packets out of large multipacket blobs and luckily it's usually quite easy to reconstruct from the static analysis.

Let's look at a small packet sent by the server when a character joins a map. Here is how packet 0xA4 is deserialised:

BinaryNinja graph view of the 0xA4 deserialiser

There are three calls to getR8BytesFromPacket, which fetch 2, 1 and 1 bytes from the decrypted buffer into the memory pointed to by RDX. Then it jumps to the end of the big deserialiser switch statement - job done.

I started off the protocol analysis by sticking to the deserialisers but eventually we have the less straightforward task of finding out how that data is used. Setting a hardware access breakpoint on the deserialised data takes us to...

BinaryNinja graph view of the function that processes paced 0xA4 data

Ugh. It's a hefty function - there's lots of work to do here and contextually the message doesn't seem important enough to be worth it at the moment - although the function is called from a few other locations so it may need looking at in the future. To get a rough idea of what the packet does we can lean on one of my favourite features of x64Dbg - display of strings from all the registers and memory locations referenced in the disassembly.



This way we don't have to figure out where interesting strings are, we can just step through the code until something meaningful pops out at us. By breaking at the highlighted location and injecting variations of the packet we can see that:

• The 2 byte value is the channel number.
• The 1 byte value is the channel type (Global, Trade, etc)
• The 1 byte after that is the language (English, German, etc)

Obviously this is horrifically unsound - a value of 0xb33f might activate the "delete System32" function and we would never know about it, but in the circumstances i'm happy to move on to one of the hundreds of more interesting packets.


Digging Deeper - Path of Exile Internals

Finding the interesting stuff requires understanding of how the server encodes the endless combinations of objects and how the client interprets them. Let's explore some of its schemes by unravelling the SRV_ADD_OBJECT message (currently 0x135), sent by the server whenever an object is added to the map.

SRV_ADD_OBJECT starts (like many other messages) with a 10 byte ID field split 4:4:2.

[00 00 03 02] [FF FF FF FF] [00 00]
   ID 0x302        -1          0

The first 4 bytes are the objects reference, this will be sent by any messages that refer to the object.

Next comes a very important 4 byte field.

D7 AA 90 65

By stepping through the processing of this random looking data we can see that it's the Murmur2 non-cryptographic hash of the string "Metadata/Characters/Str/Str" - a reference to the Marauder (ie: Strength) player character. To understand this string we have to understand the content file.

The content file

Most of the clients data is stored in an 11GB file called content.ggpk in the game directory. Fortunately for me this has already been reverse engineered with the publication of the essential PyPoE set of tools.

The Marauder entry in Characters.dat, seen using the PyPoE GGPK viewer

Much of the challenge of reversing the PoE protocol is understanding where and how the data in packets references fields in this GGPK file, and how certain fields in the file change the processing of packets. It's a good way of forcing yourself to learn how C++ data structure functions appear in assembly language.


Stats


POE uses a variable byte width encoding scheme for some of its data (such as character stats) where the first 4 bits determine how many bytes of data follow. I should probably recognise it.

0x0078 =>   0x78

0x08f0  =>   0xF0

0x9730 =>   0x1730

0xf9178d7d11 => 0x178d7d11

It wasn't too unpleasant to reconstruct from the binary (see packet_processor::customSizeByteGet() and packet_processor::customSizeByteGet_signed() in the upcoming source code) but it would be nice to know what it was.

Reverse engineering is primarily a time problem

At this stage I've run out of quick wins. The different packet ID's, different data types, items, mobile objects, different network modes (lockstep or predictive) all have variations which need custom deserialisers written and custom decoders. It's certainly a much bigger project than anticipated and I kinda want my time back to work on other things - but each packet is another thing for the todo list.

In the final post I'll talk about exileSniffer, a GUI for decrypting and dissecting Path of Exile packets which can also make its data available to build custom tools.