02 April 2012

It's Just Wrong: MSYS Python

I thought it would be helpful to have python which understand both posix path and windows path: MSYS version of python that is. In my old review about MinGW python, this kind of build already hampered with incompatibilities especially due to python heavily rely on inconsistent "defines" across modules sometime use uname() then sys.platform then posix or not (os.name).. meh.. this make workaround also inconsistent.

What did I tried is even worse:
- with uname = msys it's unknown system to python fine lets change it to cygwin, well this will create another problem since msys is not cygwin (different runtime unlike MinGW vs MSVC) its c modules should have different suffix to avoid python try to load native win32 module by accident (I set it to .myd meh...). Note: "msys" or "cygwin" or "mingw" is a hardcoded value that returned by sys.platform it can be only changed via hex editing once python.exe compiled
- distutils which used when building extension then would call uname (unlike mingw or msvc) to decide platform / compiler flags etc and what you get is insane name: MINGW32_NT-5.1-1.0.17(0.48/3/2) <- yes there even slashes there :-)
- unlike mingw build which can be changed to win32 (hardcodedly) to provide more compatibility thats not the case for msys which the only possible option is cygwin with above quirks.
- many c-extension tests failed! not really a problem if we just want barebone or purelibs though

In the end I decide to edit ntpath.py and site.py of native msvc python to make it use "/" under msys, this again have its own quirks too: win32 python need to use ";" as path separator which ":" already used as driveletter and because it doesn't know root "/". Meh...

Anyway here is the msys binaries of python 2.6.7: python-2.6.7_msys.7z (sources and patches from cygwin repo)

4 comments:

  1. Hi Tuma,

    Thank you very much for your custom Python ! It works fine and permitted me to use "waf" to compile things under MSYS.

    ReplyDelete
  2. Works great, thanks, so useful, although I did have to use rebase to fix the errors "unable to remap to same address as parent" on the .myd files, as described at the following links:
    http://www.sourceware.org/ml/cygwin/2001-12/msg00339.html
    http://www.sourceware.org/ml/cygwin/2001-12/msg00894.html

    They are talking about Cygwin but I encountered the same issue with MSYS. I resolved it with the following commands:

    mingw-get install msys-rebase
    find /usr -iname '*.dll' -o -iname '*.so' -o -iname '*.myd' >tempfile
    rebaseall -o 0x10000 -T tempfile -v

    YMMV; rebase comes with the disclaimer that there is a chance that it may completely screw up your MSYS installation. If it does get screwed up, this command should restore a working system:

    mingw-get --reinstall install $(mingw-get list | awk '/^Package:/{print $2}')

    You'll see some warnings pertaining to any packages that you have not installed; these can be ignored. Note that any changes you've made to installed files (for example, config files in /etc) will be overwritten.

    ReplyDelete
  3. Note in the rebaseall command, the -o is a minus followed by a lowercase letter O, whereas the circular characters in the 0x10000 are all number zeroes. Silly font.

    ReplyDelete
    Replies
    1. Thanks Mat, I will consider to change the font :)

      Delete