Changes between Initial Version and Version 1 of Thrashing

Timestamp:

04/02/18 18:46:00 (7 years ago)

Author:

admin

Comment:

--

Thrashing

@@ +1 @@
+= Thrashing =
+
+"Thrashing" is a condition where a computer has to continually swap information in and out of working memory to disk, because the running programs are trying to use too much memory. With Java programs like !WorldPainter a similar condition can occur where the Java code is trying to use more more memory than the configured heap size (the `-Xmx` parameter in the `worldpainter.vmoptions` or `vmoptions.txt` file) can support, causing it to spend all its time [https://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29 garbage collecting].
+
+!WorldPainter tries to avoid this but sometimes it can still happen, usually manifesting itself as the Export (or Merge) process appearing to hang while !WorldPainter is using a lot of CPU and possibly (but not necessarily) continually accessing the harddisk. It is actually making progress, but at such a slow rate that it might take weeks, months or even years to complete.
+
+Luckily there are things you can do to mitigate it:
+
+== Restart !WorldPainter ==
+
+Try this first. After a fresh start !WorldPainter is using less memory for overhead, perhaps allowing the export to proceed normally.
+
+== Reduce !WorldPainter's memory usage ==
+
+If that didn't work, you can try to get !WorldPainter to use less memory during the export, by making it export fewer regions in parallel. Normally it tries to use all your CPU cores at once, but each parallel export takes up memory and reducing the parallelisation, while making the export go slower, might avoid the thrashing condition and allow it to actually complete.
+
+=== 1. Determine how many cores it's trying to use ===
+
+The first step is to determine how many cores !WorldPainter was trying to use for the hanging export, by looking at the log file. The log file is called `logfile0.txt`, and is located in `%APPDATA%\WorldPainter` on Windows, `~/Library/Application Support/WorldPainter` on Mac OS X and `~/.worldpainter` on UNIX/Linux. Open it using any text editor and look for a line that looks like this:
+
+    Using ''n'' thread(s) for export (cores: ''x'', available memory: ''y'' MB)
+
+Make sure it is the one for the hanging export (probably the last one in the file, unless you tried more exports after that). The ''n'' is the number of cores it was trying to use.
+
+If ''n'' is 1 then it was already using no parallelisation whatsoever and there is no use in proceeding.
+
+=== 2. Configure it to use fewer cores ===
+
+To configure !WorldPainter to use fewer cores, use a text editor to open the file `worldpainter.vmoptions` (on Windows and UNIX/Linux) or `vmoptions.txt` (on Mac OS X) and add the following line:
+
+    -Dorg.pepsoft.worldpainter.threads=''n''
+
+Where ''n'' is ''one less'' than the number determined in step 1! For details about how to find and edit these files, see the MoreMemory page.
+
+=== 3. Rinse and repeat ===
+
+Save the file, (re)start !WorldPainter and retry the export. If it now works, great! If it doesn't, repeat step 2 until you reach 1 core, at which point there is no use in proceeding.
+
+'''Note:''' if you can't get it to work, then ''remove'' this parameter from the `worldpainter.vmoptions/vmoptions.txt` file again! Otherwise it will unnecessarily keep throttling !WorldPainter while you try other options.
+
+== Allocate more memory to !WorldPainter ==
+
+If nothing has worked so far you can try to give !WorldPainter more memory by following the steps on the MoreMemory page to add (or increase, if it is already there) the `-Xmx` parameter. It is important to stay well below the actual amount of physical RAM in your computer, but you can try increasing the amount by small amounts, such as 100 MB at a time. This way you may be able to fine tune it to a value which will allow the export to proceed without overtaxing your computer. Save the file and restart !WorldPainter for each try.
+
+== Export from the command line ==
+
+If none of the above worked and you're comfortable getting your hands dirty on the command line, you can try using a script to export the world instead of using !WorldPainter. Scripts use less memory than !WorldPainter proper, so they might succeed if !WorldPainter does not. Using a text editor, create a file in the directory where your .world file is stored, named `exportworld.js`, with the following contents:
+
+{{{#!javascript
+var world = wp.getWorld().fromFile(argv[1]).go();
+wp.exportWorld(world).toDirectory('.').go();
+}}}
+
+Then export the world by opening a terminal or command prompt window, navigating to the directory where your .world file is stored, and executing the following command. For more information about what the "command prompt" is and how to access and use it on Windows, see [http://dosprompt.info/ this] page. Similar information can be found for Mac OS X and UNIX/Linux. Google is your friend.
+
+{{{#!bash
+wpscript exportworld.js "MyWorldName.world"
+}}}
+
+Where `MyWorldName.world` is the name of your .world file, in double quotes in case the name contains spaces. This will export the map to a subdirectory of the current directory (which is the directory in which your .world file is stored), which you will then have to move to the Minecraft `saves` directory manually. You don't have to use the command line for that; you can drag and drop it.
+
+'''Note:''' the export script operation gives no progress feedback. This is normal, but it means there is no way to tell visually whether it is hanging or not, or how far along it is. Instead you should regularly check the total size of the files in the map directory it has created (on Windows you can do that by opening the properties of the folder in Explorer) to check whether it keeps steadily increasing.
+
+'''Note:''' with this method you cannot change the export settings from !WorldPainter's Export screen. Those settings are stored in the .world file, so you can change them by starting an export with the desired settings in !WorldPainter, immediately cancelling the export, and then saving the .world file. If you then export it with the script method the same settings will be used as for the aborted export.
+
+== Install more memory ==
+
+Of course if all else fails you can actually install more physical memory. Note that on Windows this is only likely to work for the 64-bit version. '''Important:''' after you've installed the memory, reinstall !WorldPainter so that it will actually use it!