Previous day's transcript | This day's raw log | Show all
| Time | Person | Said |
|---|
| 17:29 | ovaltofu > | pje ? |
| 17:29 | pje > | yes? |
| 17:30 | ovaltofu > | I sent you mail a while back about some hack I had to do with setuptools on linux to get it to build a regular shared library. |
| 17:30 | ovaltofu > | my message was somewhat confused (with excitement) but you never responded :) |
| 17:30 | ovaltofu > | who should I send this to ? discuss with ? |
| 17:30 | ovaltofu > | (the hack: pretend to be on darwin when importing a specific module) |
| 17:30 | gbaillie > | distutils-sig? |
| 17:31 | ovaltofu > | so setuptools is being discussed on a distutils list ? |
| 17:31 | gbaillie > | yeah ... linked from the setuptools page |
| 17:32 | gbaillie > | there's also a bug tracker |
| 17:32 | pje > | Actually, I've applied the patch in the trunk, IIRC |
| 17:32 | ovaltofu > | oh |
| 17:32 | ovaltofu > | I don't remember sending one |
| 17:32 | gbaillie > | ckdake: the links you posted are just regular html pages? |
| 17:32 | ovaltofu > | but I'll check |
| 17:33 | pje > | See http://svn.python.org/view?rev=65962&view=rev |
| 17:33 | ckdake > | gbaillie: http://fastermustache.org/fm.ics is the ics file im trying to subscribe to |
| 17:33 | pje > | Oh, wait, that's a different one. |
| 17:33 | ovaltofu > | cool, but I'm talking about a different problem |
| 17:33 | pje > | I don't recall the other thing. |
| 17:34 | ovaltofu > | sys.platform = 'darwin' |
| 17:34 | ovaltofu > | from setuptools.command import build_ext |
| 17:34 | ovaltofu > | is the hack |
| 17:34 | ovaltofu > | that way, the code that is run at import time is correct for making .so on Linux |
| 17:34 | gbaillie > | ah, sorry, i was confused, ckdake ... looking |
| 17:34 | ckdake > | gbaillie: no prob. thanks for taking a look |
| 17:34 | ovaltofu looks for the mail |
| 17:36 | ovaltofu > | pje, I just remailed you my message (2) from July 3rd |
| 17:37 | ovaltofu > | the stubs stuff is in the way |
| 17:37 | ovaltofu > | it's turned off by pretending to be on darwin which causes use_stubs=False |
| 17:37 | ovaltofu > | and all is then good |
| 17:40 | ovaltofu > | pje, why all this ? well, JCC now makes eggs that can be run from Java Tomcat |
| 17:41 | pje > | Something's not right there ovaltofu; darwin sets use_stubs = True |
| 17:41 | ovaltofu checks |
| 17:41 | ovaltofu > | right, True |
| 17:41 | ovaltofu > | my bad |
| 17:41 | gbaillie > | ckdake: you have timezones enabled (under the File menu), right? |
| 17:42 | pje > | And it sets it that way also for non-Windows platforms as long as there's a dl.RTLD_NOW |
| 17:42 | pje > | So is linux missing dl.RTLD_NOW? |
| 17:42 | ovaltofu > | yes, well, there isn't by default and that was the problem |
| 17:42 | ovaltofu > | not on the linux I was running this on |
| 17:42 | ovaltofu > | gentoo 64-bit python 2.4 |
| 17:43 | ckdake > | gbaillie: yes i do |
| 17:43 | ovaltofu > | note, that to then get the lib to really work, I have to do this from C++ later: |
| 17:43 | ovaltofu > | sprintf(buf, "libpython%d.%d.so", PY_MAJOR_VERSION, PY_MINOR_VERSION); |
| 17:43 | ovaltofu > | dlopen(buf, RTLD_NOW | RTLD_GLOBAL); |
| 17:43 | ovaltofu > | otherwise, the time extension won't link |
| 17:43 | ovaltofu > | at startup time |
| 17:43 | pje > | Hm. IIRC, the reason I did that is because if you don't eagerly load the shared library, it might not be loaded later. |
| 17:43 | gbaillie > | ok, just double-checking. |
| 17:44 | ovaltofu > | yeah, and I had to work that around elsewhere |
| 17:44 | gbaillie > | i seem to remember we do something odd with UTC datetimes ... let me look ... |
| 17:44 | pje > | i.e., say that foo_module.so is linking to bar.so |
| 17:44 | ovaltofu > | in C++, calling this myself |
| 17:44 | pje > | Hrm. |
| 17:45 | ovaltofu > | yeah, but there is no dl module on the python install I'm on (system gentoo) |
| 17:45 | pje > | Ok, doesn't really sound like a fix to me, just a hack that happens to work for your very specific circumstances. |
| 17:46 | ovaltofu > | clearly |
| 17:46 | ovaltofu > | which is why I called it a heinous hack |
| 17:46 | ovaltofu > | the bug seems to be with the expectation of a dl module to be there |
| 17:46 | ovaltofu > | when there is none |
| 17:47 | pje > | Eh? The dl usage is only included if there is a dl.RTLD_NOW |
| 17:47 | pje > | see the 'if_dl()' function |
| 17:48 | ovaltofu > | well, I can try it again without my hack give you more details on the actual error if that helps |
| 17:48 | ovaltofu goes and tries this |
| 17:48 | pje > | I assume the error would be that it builds a static lib instead of a shared one; that's how the code is written, anyway. |
| 17:48 | ovaltofu > | yes |
| 17:48 | ovaltofu > | that's the problem |
| 17:48 | ovaltofu > | (it's been over two months) |
| 17:48 | pje > | i.e., it doesn't support building shared libs except on darwin and platforms with dl.RTLD_NOW |
| 17:48 | ovaltofu > | (so I don't recall) |
| 17:49 | pje > | and in fact it technically doesn't support building shared libs at all; it's not a documented or even finished feature. |
| 17:49 | ovaltofu > | yes, and it could just do it without the RTLD_NOW |
| 17:49 | ovaltofu > | that much I know |
| 17:49 | pje > | which would then break -- at least on the Linux I used for testing. |
| 17:49 | ovaltofu > | with use_stubs=True + the later dlopen call, it works reliably |
| 17:49 | gbaillie > | JeffreyH: what's the story with UTC events? Did we have some problem where we convert them to local timezone? |
| 17:49 | pje > | If you didn't link the symbols while the current directory contained the target library, it would fail when it tried later |
| 17:50 | gbaillie > | (I'm re-reading Bug 5653 and confusing myself) |
| 17:50 | soup > | Chandler bug #5653: UTC is not being displayed correctly Product: Chandler, Component: Calendar Service, Severity: normal, Assigned to: grant@osafoundation.org, Status: ASSIGNED, Url: http://bugzilla.osafoundation.org/show_bug.cgi?id=5653 |
| 17:50 | pje > | the reason for using RTLD_NOW is to force it to find the target library while it's findable via a relative path |
| 17:50 | ovaltofu > | well, I guess it should use RTLD_NOW if it can |
| 17:50 | ovaltofu > | and still do a shared lib, if it can't |
| 17:51 | pje > | yep. and if somebody wants to put together a way to do it with optional use of ctypes, I'll review the patch. :) |
| 17:51 | ckdake > | gbaillie: that looks like it |
| 17:51 | ovaltofu > | with the caveat that then a later call (at runtime) to dlopen is going to be necessary to load all the symobls |
| 17:51 | ovaltofu > | not sure I follow, pje (ctypes) ? |
| 17:51 | pje > | using ctypes in place of the dl module. |
| 17:52 | JeffreyH > | gbaillie: since various systems export to UTC, hamstar asked (years ago) that we treat UTC special and in the calendar display the timezone for UTC events as the default timezone, so 12AM UTC would show up as 5PM. We display the event at the right time, but the UTC is kind of spurious from the user's perspective |
| 17:52 | pje > | to perform any needed dlopens from the stub loader |
| 17:52 | ovaltofu > | ah I see |
| 17:52 | ovaltofu > | you're talking at lib build time, right ? |
| 17:53 | pje > | yes, although I suppose it could also be done dynamically in the stub. |
| 17:53 | gbaillie > | ah, ok, that sounds reasonable ... let me look again @ ckdake's calendar |
| 17:53 | ovaltofu > | that's the part I don't understand |
| 17:53 | JeffreyH > | sounds like ckdake is experiencing exactly this... |
| 17:53 | ovaltofu > | the shared lib in question here is loaded without python or setuptools at all but by the JVM |
| 17:54 | ovaltofu > | so it must contain the code to do this |
| 17:54 | JeffreyH > | gbaillie: I wrote a vobject utility script that will convert UTC events to a particular timezone on import from a file |
| 17:54 | ovaltofu > | the dlopen call |
| 17:54 | ovaltofu > | and that can be done in C/C++ by adding a small C file that just calls dlopen |
| 17:54 | JeffreyH > | so if ckdake wants to do a one time import, we can use that to translate things |
| 17:54 | pje > | I'm talking about something else - I assumed we were talking about building a C extension with shared libraries also built by build_ext |
| 17:54 | ovaltofu > | yes, that's right |
| 17:55 | pje > | That's the scenario where RTLD_NOW is needed for the Python load - when the C module is imported, it needs to bind with any shared libraries there. |
| 17:55 | ovaltofu > | but the the shared lib is the one being loaded first |
| 17:55 | pje > | But it doesn't have a stub. |
| 17:55 | pje > | Only modules have stubs |
| 17:55 | ckdake > | JeffreyH: this thing is a community calendar that gets updated several times a day, so it is something i need to subscribe to. i am the admin for it so I can probably redo the way it's publishing if that helps in some way |
| 17:55 | ovaltofu > | ok, what's a stub ? |
| 17:55 | pje > | A .py file used to bootstrap the module load process. |
| 17:55 | ovaltofu > | ah yes no stub then |
| 17:56 | ovaltofu > | since the JVM knows nothing about that |
| 17:56 | JeffreyH > | ckdake: Ah. In that case, you basically want us to fix bug 5653, then. |
| 17:56 | soup > | Chandler bug #5653: UTC is not being displayed correctly Product: Chandler, Component: Calendar Service, Severity: normal, Assigned to: grant@osafoundation.org, Status: ASSIGNED, Url: http://bugzilla.osafoundation.org/show_bug.cgi?id=5653 |
| 17:56 | ckdake > | JeffreyH: sounds like it :) |
| 17:56 | pje > | So are you building extensions or just a shared lib? What is this workaround *for*? |
| 17:57 | gbaillie > | JeffreyH, I don't think what you described is what I'm seeing. |
| 17:57 | ovaltofu > | I'm building a python extension, JCC, that also includes a shared lib |
| 17:57 | gbaillie > | i.e. the VEVENT has DTSTART;VALUE=DATE-TIME:20080928T113000Z |
| 17:57 | ovaltofu > | I'm using the convenience of setuptools to build the shared lib |
| 17:57 | pje > | ok. so when the extension is loaded, it needs to force the shared lib to load |
| 17:57 | ovaltofu > | that shared lib is loaded either by the extensions JCC builds or by other things embedding python, such as a JVM |
| 17:57 | gbaillie > | oh, no, it's correct on the calendar |
| 17:57 | gbaillie > | nm |
| 17:58 | JeffreyH > | it shows up at the right time, but displays 11:30 instead of 4:30AM, right? |
| 17:58 | JeffreyH > | (or whatever, depending on where you've set your tz to) |
| 17:58 | gbaillie > | yeah, that's right, JeffreyH |
| 17:58 | ovaltofu > | no extension is loaded |
| 17:58 | ovaltofu > | the shared lib loads python and initializes it |
| 17:58 | ovaltofu > | that's when it breaks, without the dlopen call |
| 17:59 | pje > | Which is why it's not supported. |
| 17:59 | ovaltofu > | but it could be |
| 17:59 | pje > | We seem to be going in circles. |
| 17:59 | ovaltofu > | now I understand what you mean |
| 17:59 | ovaltofu > | it could be with use_stubs=True |