[BUG] Prusa MK2.5S firmware 3.10.0 (with MMU2S 1.0.6) randomly repeats a short section of gcode several times before failing with "static memory overwritten" error

[IntegersOfK]

Printer type - MK2.5S with MMU2S
Printer firmware version- 3.10.0-4481

MMU Upgrade - MMU2S
MMU upgrade firmware version - 1.0.6-372

SD card or USB/Octoprint
SD Card

Describe the bug
Printer has an odd movement, then repeats a short section of the gcode and eventually fails entirely with "static memory overwritten" error. If you restart the printer and do the print again, with the same print file, the failure may happen in a different section/different time.

After getting this error multiple times (in different sections of the same print), for my troubleshooting I reset the printer and MMU to factory defaults and reflashed the latest firmware on both. I did the calibration wizard all fresh and am using PrusaSlicer to create the gcode. This is a single colour print (using PrusaSlicer default profile "Single").

I have seen the problem occurs on both of these files:
Fabric_of_Thyme_2.0_10x10_0.35mm_PLA_MK2.5SMMU2S_3h37m.zip

I have replaced the SD card as a last resort in case there was some kind of read error or corruption issue with that, using the same gcode files exported from PrusaSlicer. It does not appear to be related to the SD Card.

It's worth noting that after I see the issue happen, I can pause the print from the menu and when it resumes it appears to continue until the issue happens again eventually in another section or on the next print of the same file (but again, not always in the same place).

Here is a picture which shows the section of gcode has repeated itself several times before shutting down entirely:

Video
https://youtu.be/z2-jaG_GTXU

The problem was also happening on the 3.10.0 RC1, but I had never had it happen on on earlier firmware versions (3.9.3) though I did not try these specific gcodes on that firmware version (I will reflash older firmware and try it now though).

#2964 is another issue which appears to mention this type of error, though that issue report it happening at the start of the print.

#2791 appears to be somewhat related to solving a similar issue, but is related to sensors?

I think some changelog referenced "strange movements" before this error, which feels close to this problem too.

[wavexx]

Using 3.10 could you try to disable the filament sensor (settings -> filament sensor [off]) and see if that happens again or if the problem disappears?

[IntegersOfK]

I would like to clarify that with 3.10 I am not noticing the issue on every print, even when using the same gcode. In particular, after I flash the firmware fresh to 3.10, it always seemed to handle the first print normally, but when I retry the same gcode again right away, I get the error eventually somewhere different each time. That intermittency/inconsistency in when it occurs, is probably going to make it difficult to give a full overview of how to reproduce it, though I can say on 3.10 it almost always happening the second and subsequent prints after flashing the firmware (which might be a coincidence that the first one has always worked after flashing, I don't know if it was worth mentioning, just something I noticed). Again, always using that same gcode, the 10x10 "fabric" link model.

Using 3.10 could you try to disable the filament sensor (settings -> filament sensor [off]) and see if that happens again or if the problem disappears?

That's a good suggestion, I will give it a try next! Today I went back to 3.9.3 with the same gcode and filament, and have done two copies of the same print now without getting that error. I would like to do a few more with 3.9.3 due to that intermittency I mentioned, just to confirm that we really aren't seeing it happen on 3.9.3 and I think the sample size needs to be higher to say it with confidence. After a few more and I'm satisfied that it's not an issue on 3.9.3, I will reflash 3.10, wait for the problem one more time, and then disable the filament sensor for some testing after that. Will report back (probably in a day or two) with those results.

[wavexx]

From the looks of it, it could be either some memory corruption issue (triggered by timing more than gcode IMHO), or some (incorrect) repeated call to the pause/resume machinery in an incorrect moment. Disabling the filament sensor mostly removes one source that could cause pause/resume internally.

These are nasty to debug. Any detail/step you can provide is useful. FW 3.10 is actually faster in a few places and uses less memory as well, which incidentally can cause an old timing bug to surface now, or a memory overwrite bug to overwrite something which is now critical :(

[wavexx]

As an additional note, to help making the bug more reproducible, try resetting the printer at each print. This ensures the starting conditions are more similar and could result in a more reproducible scenario.

[IntegersOfK]

After several more prints of the same gcode yesterday on 3.9.3, I feel confident that 3.9.3 does not contain the bug. Later today I will reflash 3.10, wait for the error, then disable the filament sensor and try again a few times. More to come!

[leptun]

The SD card reading was changed in 3.10 to be more efficient. Maybe something broke along the way. There have been other issues relating to this that were fixed in the RCs, but maybe there is still some edge case that wasn't encountered so far.

[IntegersOfK]

I turned off the filament sensor before starting the print again with 3.10, and I am sorry to report that I encountered the error again (in a different place as previously described) with that same print. I confirmed after the strange repeat movement started that the filament sensor was indeed off the whole time.

The SD card reading was changed in 3.10 to be more efficient. Maybe something broke along the way. There have been other issues relating to this that were fixed in the RCs, but maybe there is still some edge case that wasn't encountered so far.

Should I install OctoPrint on my Raspberry Pi and try a print from that? Would that be a valid test to determine if it's SD-card related? As I said before, I have already tried two different brands of SD card, so I really don't think it could be the card itself.

[IntegersOfK]

I did two prints with OctoPrint and did not get the error. I will try a few more tomorrow and turn the filament sensor back on. But preliminary testing indeed feels like an SD Card read issue then I suppose. Will report back again after I can confirm a few more with OctoPrint.

[IntegersOfK]

After a few more prints, it works fine from OctoPrint. Same gcode, which I even copied from the SD card directly in case there was some kind of corruption that I was missing. So I guess @leptun is onto something when suggesting it could be the SD read logic. Is there anything else I can do to test for you about this? Are there logs anywhere I can record while it happens maybe?

[djakkk]

I have the same problem after updating my MK2.5 to 3.10.0 (no MMU2 connected)

The first print I tried ran to about 95% and then the printer started the gcode over from line 1 with bed leveling.
The 2nd time I tried the same print it did the exact same thing.

When I tried to print another file I got the "static memory overwritten" error"

After that I ran a factory reset (clear all) and flashed the firmware again (the same version)
After running the Wizard, first layer calibration and trying the same files from the same SD-card (a Kingston 8GB Class 4)

This time the 1st print (15 min) worked, but the 2nd print (30 min) only ran for about 40% before the printer restarted the gcode.

After this I copied the gcodes to a new SD-card (Verbatim 4GB Class 10), but this time the 2nd print only ran for about 5% untill the printer restarted the gcode.

So now the printer is useless... not sure if I should downgrade the firmware, wait for a fix, or try using octoprint.

[IntegersOfK]

not sure if I should downgrade the firmware, wait for a fix, or try using octoprint.

Well I can confirm that both options will work in the meantime. 3.9.3 has no issues, and OctoPrint has been working great.

Obviously I agree that this is a frustrating issue and absolutely must be fixed (otherwise we can't print from SD Card). Let's give them some time, I'm confident with our detailed reporting, the Prusa team and contributors will be able to solve it.

[djakkk]

Well I can confirm that both options will work in the meantime. 3.9.3 has no issues, and OctoPrint has been working great.

Obviously I agree that this is a frustrating issue and absolutely must be fixed (otherwise we can't print from SD Card). Let's give them some time, I'm confident with our detailed reporting, the Prusa team and contributors will be able to solve it.

I'm gonna dig out an old Raspberry pi and install octoprint and see if that helps here too.
I hope so, any idea if the MK3 has the same problem?

[djakkk]

Just printed the 30min gcode via octoprint and that worked perfect, so the issue seems to be with the SD-cards.

[DRracer]

Sadly we haven't been able to reproduce the issue so far (2kg of filament ;) ):

From your reports it looks like something related to reading from the SD card + doing a second print (may be even stop the first print and then start another one).