Chandler wxPython v2.x.x.x Migration Detail

David Surovell

Chandler GUI Frameworks Engineer, OSA Foundation

27 July 2005

Status:

1. v2.6.2.0 integration is underway
   • remaining an ongoing process
2. completed v2.6.1.0 integration
   • converted integration to gradual ongoing process
3. completed v2.6.0.0 integration
   • (01 May 05) M2Crypto needs a SWIG v1.3.24 update
   • (01 May 05) SWIG v1.3.24 Windows build needs some fixes
4. completed v2.5.3.1 migration
5. see related WxPythonUpdatePlan
   
   
   
   

Questions:

(unanswered)

1. none for the moment

(answered, for the moment)

1. makefile, line 110 (7-8 lines) - wxWidgets: $(SRC)/$(BUILD)/Makefile - does this need to be changed for monolithic build?
   
   No, it is the same. The monolithic build only affects the core lib(s), the contrib libs are still separate. --Robin
   
   
   
   
2. how should "Chandler.py" be changed regarding wx.appl initialization ordering?
   
   (refer to the wxPython v2.5.2.x Migration documentation)
   
   Nothing should change. If it was working with v2.5.1 then it still will with v2.5.2. The only difference is that exceptions will now be raised when trying to do something that you shouldn't. --Robin
   
   
   
   
3. There exists a directory "/external/debug/include/" which contains many (if not all) wxWidgets C++ header files. In particular, there exists an older "version.h", containing a "v2.5.1" value. How is this file being used?
   
   If it still has 2.5.1 then something has gone wrong with the "make install" step of the wxWidgets build. --Robin
   
   
   
   
4. is the "framework/.../wx/__init__.py" forcing the search for the non-existent dylib files?
   
   No, it will load _core _.so (and the other _* _.so extension modules) which will in turn load whatever dylib files they were linked with. You can use the otool command to get a list of the dylibs that each is trying to load, for example "otool -L _core _.so --Robin
   
   
   
   

Battle Recap (v2.6.0.0)

ACTION(S): performed a QD integration of wxPython v2.6.0.0 final distribution

RESULT(S): contrib/animate doesn't build

contrib/animate/mac/animate_wrap.cpp:1381:32: wx/animate/animate.h: No such file or directory

gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -g -Wall -Wstrict-prototypes -g -DSWIG_TYPE_TABLE=_wxPython_table -DHAVE_CONFIG_H -DWXP_USE_THREAD=1 -UNDEBUG -D__WXDEBUG__ -D__WXMAC__ -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -DNO_GCC_PRAGMA -Iinclude -Isrc -I/Users/davidsurovell/ChandRoot/external/debug/lib/wx/include/mac-unicode-debug-2.6 -I/Users/davidsurovell/ChandRoot/external/debug/include/wx-2.6 -I/Users/davidsurovell/ChandRoot/external/debug/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c contrib/animate/mac/animate_wrap.cpp -o build_debug.unicode/temp.darwin-7.9.0-Power_Macintosh-2.4/contrib/animate/mac/animate_wrap.o -g -O0

CONCLUSION(S): fixed - added 2 build lines to the makefile

-->

ACTION(S): performed a QD integration of wxPython v2.6.0.0 final distribution

RESULT(S): PyLucene and SWIG v1.3.24 don't get along

/Users/davidsurovell/ChandRoot/external/debug/bin/swig -modern -DSWIG_COBJECT_TYPES -DPYLUCENE_VER="'0.9.8-189'" -DLUCENE_VER="'1.4.3'" -D_WITH_DB_DIRECTORY=1 -DDB_LIB_VER="'4.3'" -DDB_VER=43 -Idebug/classes -c++ -python PyLucene.i
/Users/davidsurovell/ChandRoot/external/gcc-3.4.3/bin/g++ -shared -bundle -o debug/_PyLucene.so -O0 -g  -DSWIG_COBJECT_TYPES -DPYLUCENE_VER="'0.9.8-189'" -DLUCENE_VER="'1.4.3'" -D_WITH_DB_DIRECTORY=1 -DDB_LIB_VER="'4.3'" -DDB_VER=43 -I/Users/davidsurovell/ChandRoot/external/debug/db/include -I/Users/davidsurovell/ChandRoot/external/persistence/db/db-4.3.27 -I/Users/davidsurovell/ChandRoot/external/persistence/db/db-4.3.27/build_debug -I/Users/davidsurovell/ChandRoot/external/gcc-3.4.3/include -Idebug/classes -I/Users/davidsurovell/ChandRoot/external/debug/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 PyLucene_wrap.cxx debug/store.o debug/db.o debug/lucene.o debug/snowball.o debug/highlight.o debug/analyzers.o debug/unreleased.o debug/util.java.o debug/util.cpp.o debug/io.java.o debug/io.cpp.o debug/store.java.o debug/store.cpp.o debug/search.java.o debug/search.cpp.o debug/queryParser.java.o debug/queryParser.cpp.o debug/analysis.java.o debug/analysis.cpp.o debug/highlight.java.o debug/highlight.cpp.o -L/Users/davidsurovell/ChandRoot/external/gcc-3.4.3/lib -lgcj -liconv -undefined suppress -flat_namespace -multiply_defined suppress

PyLucene_wrap.cxx: In function `void initializeStaticMembers()':
PyLucene_wrap.cxx:2351: error: `swig_globalvar' undeclared (first use this function)
PyLucene_wrap.cxx:2351: error: (Each undeclared identifier is reported only once for each function it appears in.)
PyLucene_wrap.cxx:2351: error: `var' undeclared (first use this function)
PyLucene_wrap.cxx:2351: error: `swig_varlinkobject' undeclared (first use this function)
PyLucene_wrap.cxx:2351: error: expected primary-expression before ')' token
PyLucene_wrap.cxx:2351: error: expected `)' before "cvar"
PyLucene_wrap.cxx:2355: error: `cls' undeclared (first use this function)
PyLucene_wrap.cxx: At global scope:
PyLucene_wrap.cxx:48972: error: `struct swig_globalvar' used prior to declaration
PyLucene_wrap.cxx:48977: error: `typedef struct swig_globalvar swig_globalvar' used prior to declaration
PyLucene_wrap.cxx:48979: error: `struct swig_varlinkobject' used prior to declaration
PyLucene_wrap.cxx:48982: error: `typedef struct swig_varlinkobject swig_varlinkobject' used prior to declaration

CONCLUSION(S): fixed by Andi Vadja - PyLucene needed an update

-->

ACTION(S): performed a QD integration of wxPython v2.6.0.0 final distribution

RESULT(S): external "make DEBUG=1 clean" fails in vobject and a few other places

CONCLUSION(S): transitory problem

-->

ACTION(S): TBD

RESULT(S): TBD

CONCLUSION(S): TBD

Battle Recap (v2.5.3.1)

ACTION(S): perform a proper diff between OSAF CVS and new official wxPython v2.5.3.1 source trees

• diff -U 8 -r "cvs-subtree" "tree-new-non-cvs" > diff.log
• grep -a "Only in" diff.log
• separate CVS-local and new v2531 filenames and paths into 2 text files
• resolve crud entries: ".DS_STORE", checkout orphans, etc.

RESULT(S): in progress

CONCLUSION(S): TBD

-->

ACTION(S): TBD

RESULT(S): TBD

CONCLUSION(S): TBD

Battle Recap (v2.5.2.9p4)

ACTION(S): initial effort (see Update Plan document)

• rename "internal/wxPython2.5" (currently v2.5.1) to "internal/wxPython2.5-old"
• create new one "internal/wxPython2.5" wxPython v2.5.2.8 distribution
• copy "makefile" and "__hardhat__.py" from the old sub-tree into the new one

RESULT(S): build failure ld: undefined - wxSashLayoutWindow::ms_classInfo

CONCLUSION(S): link looks to be missing "libwx_mac_adv-2.5.dylib"

-->

ACTION(S): edit "wxPython2.5/makefile" (2x) changed "--disable-monolithic" to "--enable-monolithic"

RESULT(S): build failure (although more stuff now builds)

gcc
   -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic
   -g -Wall -Wstrict-prototypes -g -DSWIG_GLOBAL -DHAVE_CONFIG_H -DWXP_USE_THREAD=1 -UNDEBUG
   -D__WXDEBUG__ -D__WXMAC__ -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -DWX_PRECOMP -DNO_GCC_PRAGMA
   -Iinclude -Isrc -I/Users/davidsurovell/ChandRoot/external/debug/lib/wx/include/macd-2.5
   -I/Users/davidsurovell/ChandRoot/external/debug/include/wx-2.5
   -I/Users/davidsurovell/ChandRoot/external/debug/Library/Frameworks/Python.framework/Versions/2.3/include/python2.3
   -c src/libpy.c -o build_debug/temp.darwin-7.5.0-Power_Macintosh-2.3/src/libpy.o -g -O0

gcc
   -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic
   -g -Wall -Wstrict-prototypes -g -DSWIG_GLOBAL -DHAVE_CONFIG_H -DWXP_USE_THREAD=1 -UNDEBUG
   -D__WXDEBUG__ -D__WXMAC__ -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -DWX_PRECOMP -DNO_GCC_PRAGMA
   -Iinclude -Isrc -I/Users/davidsurovell/ChandRoot/external/debug/lib/wx/include/macd-2.5
   -I/Users/davidsurovell/ChandRoot/external/debug/include/wx-2.5
   -I/Users/davidsurovell/ChandRoot/external/debug/Library/Frameworks/Python.framework/Versions/2.3/include/python2.3
   -c src/helpers.cpp -o build_debug/temp.darwin-7.5.0-Power_Macintosh-2.3/src/helpers.o -g -O0

c++
   -Wl,-F. -bundle -framework Python
   -g build_debug/temp.darwin-7.5.0-Power_Macintosh-2.3/src/helpers.o build_debug/temp.darwin-7.5.0-Power_Macintosh-2.3/src/libpy.o
   build_debug/temp.darwin-7.5.0-Power_Macintosh-2.3/src/mac/_core_wrap.o -L/Users/davidsurovell/ChandRoot/external/debug/lib
   -lstdc++ -lwx_macd-2.5 -o build_debug/lib.darwin-7.5.0-Power_Macintosh-2.3/wx/_core_.so
   -framework QuickTime -framework Carbon -framework Cocoa -framework System

ld: Undefined symbols:
__Py_Dealloc
__Py_NegativeRefcount
__Py_RefTotal
_Py_InitModule4TraceRefs
__PyObject_DebugMalloc

CONCLUSION(S):

• spoke to Andi V., who identified problem as "wxPython/distutils" version conflict

-->

ACTION(S): remove "wxPython/distutils" sub-tree from new distro

RESULT(S): build runs, but appears to be using wxWidgets v251 dylibs

CONCLUSION(S): delete v251 dylib files

-->

ACTION(S): delete all wxWidgets v251 dylib files

RESULT(S): no longer uses v2.5.1 dylibs, but fails on launch

Traceback (most recent call last):
  File "Chandler.py", line 10, in main
    import sys, logging, traceback, wx
  File "/Users/davidsurovell/ChandRoot/external/debug/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wx/__init__.py", line 19, in ?
    # Sub-packages
  File "/Users/davidsurovell/ChandRoot/external/debug/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wx/core.py", line 4, in ?
    import _core
ImportError: Failure linking new module:
- /Users/davidsurovell/ChandRoot/external/debug/lib/libwx_macd_html-2.5.1.dylib:
- dyld: /Users/davidsurovell/ChandRoot/chandler/debug/Library/Frameworks/Python.framework/Versions/2.3/Resources/Python.app/Contents/MacOS/Python
- can't open library: /Users/davidsurovell/ChandRoot/external/debug/lib/libwx_macd_html-2.5.1.dylib  (No such file or directory, errno = 2)

Traceback (most recent call last):
  File "Chandler.py", line 59, in ?
    main()
  File "Chandler.py", line 46, in main
    dialog = wx.MessageDialog(None, message, "Chandler", wx.OK | wx.ICON_INFORMATION)
UnboundLocalError: local variable 'wx' referenced before assignment
[49762 refs]

CONCLUSION(S): TBD

-->

ACTION(S): edit "wxPython2.5/makefile"

• added line 58 - --enable-html * removed line 104 - "$(SNAP)/lib/libwx_base_carbon* \"

RESULT(S): no change -> fails on launch

CONCLUSION(S): change not necessary - backed it out

-->

ACTION(S): added a "libwx_macd_html-2.5.1.dylib"

RESULT(S): fails to load another wx dylib - "...adv..."

CONCLUSION(S):

• problem isn't specific to HTML library
• spoke to Heikki T., who suggested that part of the problem may be related to lack of fully-clean "external" sub-tree
• dispose of entire Chandler tree, re-checkout from CVS and re-try from first battle step without building

-->

ACTION(S): perform complete clean source fetch and rebuild (3 hours!)

• prepared wxPython sub-tree as before, but prior to executing any build steps
• CAVEAT: make risky, bad choice to switch over to first preview release of wxPython v2.5.2.9

RESULT(S):

• application launches: Chandler splash window displayed, parcels loaded, then crashed in Thread 0 at wxWindow::Freeze()

CONCLUSION(S):

• build hell problems likely finished
• crash possibly caused by some wxPython v2.5.2.9 instability, perhaps the known WxGrid problem
• ideally would have waited for more stable release of v2.5.2.9

-->

ACTION(S): fixed wxWidgets wxWindow::Freeze/Thaw NULL-derefs

RESULT(S): Chandler launches safely, but...

• main window has missing sub-views and content
• various wxDC error messages are logged

CONCLUSION(S):

• fix Chandler sources that call wxDC improperly
• send email to wx-dev list regarding problems

-->

ACTION(S): fixed various wxDC calls in Chandler: Blit, DrawLine, DrawRectangle, DrawText

• 2 calls in "chandler/parcels/osaf/framework/attributeEditors/AttributeEditors.py"
• 8-10 calls in "chandler/application/SimpleCanvas.py"
• 8-10 calls in "chandler/parcels/osaf/framework/blocks/calendar/CalendarCanvas.py"
• 4-5 calls in "chandler/parcels/osaf/framework/blocks/calendar/CollectionCanvas.py"
• 3-4 calls in "chandler/parcels/osaf/framework/blocks/calendar/CalendarBlocks.py"

RESULT(S):

• more main window content is rendered
• calendar works better now
• Mac: main window is missing window divider sashes, but drag "dot" IS visible and draggable

CONCLUSION(S): time to work on Win32

-->

ACTION(S): Win32-cygwin build fails

• copy OSAF-authored MSVC v7.1 project files back into new project tree
• edit "WxPython2.5/makefile":
  • line 34 - change "251" to "252"
  • line 39 - add "BUILD_ACTIVEX=0"
• edit old "setup.h": add "#define wxUSE_MDI 1"
  • there are more changes!! Review and document ALL other changes
  • copy "setup.h" to 2-3 other places in build (TBD)

RESULT(S): Win32 build works

CONCLUSION(S): TBD

-->

ACTION(S): TBD

RESULT(S): TBD

CONCLUSION(S): TBD