The 1.3.4.3 Method Rewrite Post

While I was taking my Uber back from the Postmodern Jukebox concert, I figured out how I was going to "fix" Terraria and allow myself to release an update for RomTerraria at the same time: I'm going to rewrite the method that is introducing the issue.

I've patched bugs in the base game before. For the longest time, if you didn't have a My Games/Terraria folder in your Documents folder, the game would crash when trying to create the settings file.  I patched that with the original releases of RomTerraria.

Fortunately, I've got a good place to handle injecting my replacement method.  LiquidRenderer.PrepareDraw just calls the internal method.

I'm going to do this patch in two phases.

Phase one will be out tomorrow on Thanksgiving assuming nothing else blows up and will simply make PrepareDraw be a no-op.  This will allow everything else to work as is EXCEPT for the new water effects.

Phase two will take significantly longer.  I'm going to rewrite LiquidRenderer.InternalPrepareDraw.  It's a bit over 400 lines of, well, not good code.  I mean, it's dereferencing to memory before a pointer.  If I was going to be a total dick, I could craft a world file that could potentially turn into a native code exploit.  Fortunately, I'm not a total dick.  This will take me a lot longer to do and may end up moot if they fix the issue before my rewrite is done.

I've only got this weekend to do this, so here goes nothing.

Update 11/24/2016, 11:57am: Well, it's not going to happen this morning, but I am working on it...

Update 11/24/2016, 12:20pm: I've gotten the replacement class building as a standalone DLL.  I've got to do IL-patches to replace 14 instances of the class in six methods in two classes.  I'm going to try to get that done before I have to leave for my afternoon social obligation.

Update 11/24/2016, 4:38pm: I'm back from my social obligation and am resuming my coding.  I've almost got the replacement working.

Update 11/24/2016, 9:57pm: Stopping for the night.  I've got my replacement class patched in and am working on debugging, but I'm exhausted.  That said, making PrepareDraw a no-op has other side effects and isn't a good solution, so I'm dropping that part of the plan.

Update 11/25/2016, 12:18pm: Taking a lunch break.  Since I'm afraid I'm going to have to do this kind of thing again in the future, I've created a general purpose class replacement system. Will resume testing once I am fed.

Update 11/25/2016, 1:03pm: I'm now able to reproduce the crash inside my custom class inside the debugger.  Should help me fix it.

Update 11/25/2016, 1:09pm: Well, it's good that I made the class replacement system generic, because the crash is NOT caused by the code I suspected.  It appears that something else is running amok around memory.  Give me a bit of time to figure out the appropriate memory layout so I can figure out which object is immediately before this one in the heap.

Update 11/25/2016, 1:30pm: I've got a 1.2GB memory dump that I'm manually going through.  I'd be going faster, but the .NET memory tools require a minimum .NET version of 4.5 and Terraria is compiled to 4.0.

Update 11/25/2016, 2:25pm: Good news: managed to stop the game from crashing. Bad news: filled blocks of water aren't rendering.  Still going.

Update 11/25/2016, 3:53pm: I've checked my current code into Github.  Still trying to figure out why full blocks don't render.  Note that this doesn't truly fix the memory overwrite, but it leaves the old 160KB object around as a "safety buffer."  I'm going to try for another hour, then call it a day.  I've still got work to do around the house.

Update 11/25/2016, 5:51pm: Part of the problem is that the version of Telerik JustDecompile that I'm using doesn't properly decompile unsafe code evidently.  Testing a fix now.

Update 11/25/2016 6:00pm: Adding padding around the wave bank let me run with water effects for almost ten seconds before the game crashed in a spectacular fashion.  Looks like something is hardcoded with a max resolution and when you go above that resolution, it starts walking over memory it shouldn't.  Dinner shortly, then more research

Update 11/25/2016 6:09pm: Here's a test version.  This is not guaranteed to work, but appears to be working in my limited testing.  Please let me know if it crashes for you.

Update 12/6/2016 1:13pm: v1.3.4.4 was just released.  The test version was compiled and tested against v1.3.4.3, and may not work with v1.3.4.4.  I'm at work for several more hours.  Let me know if I need to address anything after the fix.

Update 12/19/2016 9:35pm: Added LAA support code.  Trying to fix an issue with rendering water that surfaced at certain resolutions for v1.3.4.4 before releasing an update.

Update 12/25/2016 3:58pm: Build for v1.3.4.4 here.  You'll have to set up your settings again.

Update

I'm currently at a concert for Postmodern Jukebox and won't be home until almost midnight.

I've got tomorrow morning blocked out for rewriting the defective method in Terraria that's doing a negative address pointer dereference and swapping it out.  I've patched the base game before.  I'll do it again...

1.3.4 Support Coming

Sounds like Terraria v1.3.4.x broke RomTerraria again.

If I had to make a guess, I'd say that the new render pathway for water changed the number of RenderTarget2D's in the code.  If that is the case, it's probably a 20-30 minute fix.

That said, I haven't had a chance to look at it because I've been working, and I won't be getting home from work until late tonight.

I'll take a look at it after dinner tonight.

Update 11/16/2016, 8:23pm: I've got the resolution patch working, but the new water effects are causing the following crash that I need to fix first...

...removed for space...

Update 11/22/2016, 11:10am: Patch appears to be working, but I have to finish my test pass before I release it.  Hopefully tonight after dinner.

Update 11/22/2016, 6:41pm: I found a crash bug with 1.3.4.3. Working on patching it now.

Update 11/22/2016, 8:24pm: Okay, I've got a choice.  I can either release a version that will only work with all the visual bells and whistles turned off, or I can hold back and release it once I figure out what is causing the wave mask to get corrupted and crash approximately ten frames into rendering.  Right now, I'm planning on holding back until I get this tracked down.

I know that isn't what people want to hear, but I'd like you to remember two things.  First, the reason that RomTerraria works as well as it does across multiple versions of the game is that I don't do source decompiling/recompiling.  I do MSIL-level opcode patching.  This requires a lot more time to do right because if I do it wrong, I create invalid code and that breaks everything else.

Second, the reason that you never see anything about this mod over on anything official is because back when Terraria launched, some asshats were using my mod as a crack for Terraria and they bundled my mod with the cracked version.  Because of this, Re-Logic appears to see this mod as something bad.  As someone who spent a decade in the games industry fighting pirates, I can't blame them.  I'm absolutely certain that they see me as someone who has enabled bad behavior and they don't want to enable me to succeed in this venture.

I've been maintaining this mod as a spare-time project for over five years now, and it wears on me.  I don't make any money off of it, partly because I don't want to further enrage Re-Logic, and partly because it doesn't seem right to me to make money off of mods.  However, that means that this mod is secondary to other projects at both work and home.  Even tonight, I've had to spend time helping out a lady friend.

I spend the time that I can afford to on this, both from a financial point of view, but also a mental point of view.  Working on this mod isn't as simple as hitting F5 in Visual Studio and seeing what happens.  It requires a lot of time to trace through each failure.  It requires a lot of effort to inject diagnostics code into the assemblies.  It requires a lot of patience to track down crashes, support comments, etc.  I have a finite amount of willpower to get through each day and this mod gets the last few minutes of the day usually.

I'll keep working until I head to bed tonight, but I won't be able to work on it again until Thursday night or Friday morning at the earliest due to the holiday and social obligations.  I'm off Friday, however, so hopefully I can refocus my energies for a bit and get this fixed.

Update 11/22/2016, 8:50pm: Around 40 frames in, something is corrupting Terraria.GameContent.Liquid.LiquidRenderer._waveMask and changing it from an array of 40000 XNA Color structs to an array of 39936 XNA Color structs.  This is going to take a lot longer to figure out since Terraria uses unsafe code to work with that array.

Update 11/22/2016, 9:05pm: Well, the good news is that I figured out the root cause.  There's a bad bug in LiquidRenderer.InternalPrepareDraw when it's dealing with a fixed pointer to _waveMask.  This code has a lot of issues.  It's not cache coherent (looping over Y inside of a loop over X), and the root cause is a chunk of code that starts writing before the variable if you are on a screen with certain conditions set:

*(vISCOSITYMASK - 200 * sizeof(Color)) = *vISCOSITYMASK;

They're trying to set the pixel above a point to the value of that point.  That line should scare you.  Scares the hell out of me.  Until I can trace through what the hell led to this, or Re-Logic writes a cache coherent version of this method that doesn't write to memory before vISCOSITYMASK pointer, there will not be an update to RomTerraria.

Update 11/26/2016, 12:57am: A test version that seems to work with 1.3.4.3 is available over at the update post.