AW: SETUP WIZARD FOR CYGWIN?XFREE

[Charles Wilson]

Harold Hunt wrote:
> Ralf,
> 
> Just so you know, I understand and completely support what you are doing.
> KDE on Cygwin is not a stable port yet, it is a work in progress, and you
> are following the mantra of 'release early, release often'.  In addition,
> you are being smart by including additional DLL versions that you know to
> work with KDE.  I wouldn't worry about 'support problems' when releasing
> beta packages of a free software project; it should be quite clear that the
> user is on his/her own regarding support.
> 
> Great work, and keep it up,

I agree, it is great work and I fully support Ralf -- he's going a great
job. I also don't have any problem with him providing known stable
versions of the "my" dll's in a beta dist of his KDE stuff.  My problem
is that this thread started with a discussion of an *installer* for the
cygwin/xfree/kde.  My natural assumption was that Ralf was moving
towards "official" -- that is, non-beta -- dists of kde(1? 2?).

THAT's what got my dander up.  If I misunderstood, then I apologize.

> If you say that, I will try it, although I have had much trouble with the cygwin
> dll
> (socket problems) and my confidence is a little corrupted because of initial
> ignoring
> my cygwin bug report. 

I understand.  There are very few true cygwin-kernel hackers (e.g. I'm
not one...not really).  They are currently overwhelmed, and stuff falls
thru the cracks (the same thing is true on any large project -- I've had
patches to binutils@ fall off the radar.  Even DJ Delorie !! has had
stuff "ignored" on THAT list).  

However, libpng != cygwin.  "My" dll's are mere ports of stuff that
typically have large groups of bug-fixing hackers.  When new releases of
"my" packages are available from the originating groups, I usually
update quickly -- and almost always with an extended "[avail for test]
....." period before making them "official".  However, I only send the
[avail for test] messages to the main cygwin list; perhaps I should also
copy the cygwin-xfree list so you guys don't miss the chance to report
regressions before it's "too late"?

> Additional I have had trouble with ncurses lib which was
> linked with
> cygwin 1.3.1 and don't work with cygwin 1.1.8 because of using the
> check_for_executable()
> function.

Well, that was an ugly situation all around.  However, it's part of the
price we pay.  backwards compatibility of cygwin1.dll is guaranteed
(until cygwin2.dll if ever) but *forward* compatibility is not.  That
is, you should be able to replace cygwin1.dll-1.1.8 with cygwin1.dll-(x
>= 1).(y >= 1).(z >= 8) -- but not necessarily vice versa.  (And with multiple dependencies comes
versioning problems.  Ever try to rpm -Uvh evolution-X-X.rpm on linux? 
NIGHTMARE. There was
even a story about the problem with GNUcash on Slashdot and LWN:
http://slashdot.org/articles/01/06/15/0128247.shtml
http://lwn.net/2001/0614/ )

The cygwin developers TRY to maintain forward compatibility when
possible -- and in fact, the 1.3.1 problem you mention was a mistake. 
The check_for_executable should have been encapsulated inside an opaque
structure like __impure_ptr, but it was overlooked.  Anyway, I don't
fully understand, but I believe that if I recompiled ncurses NOW under
1.3.2, that check_for_executable() wouldn't show up as a linked
(required) symbol.  However, ALL future cygwin1.dll will still *export*
the symbol, so that apps unlucky enough to have been compiled under
1.3.1 will still work with future kernels.  (Those unlucky apps *won't*
work with earlier kernels, because the earlier kernels don't have the
symbol -- as you discovered).

OTOH, 1.3.1 WAS released over three months ago.  cygwin-xfree ALWAYS
requires the most recent cygwin1.dll.  Why is it so terrible to also
require >=1.3.1 for ncurses?

As far as "ignoring" your initial bug report, that's a little harsh (and
inaccurate) IMO.  The cygwin list at least 50 messages a day, sometimes
more than 100.  sometimes things fall thru the cracks -- be persistent. 
You were -- and your problems were (eventually) addressed.  as cgf said:
> If something is broken in 1.3.2, we will try to fix it.

[side note: There have been a number of important fixes post 1.3.2, but
right now the CVS version is...err, flaky.  Perhaps "the cygwin folks"
ought to go back to a known good date, and release something and call
THAT "1.3.3" -- but that would probably distract the developers from
fixing the current problems and releasing what we're currently referring
to as "1.3.3" sooner) ]

> This seem to me, that sometime patches are made and released which have some
> unknown side affects. Don*t misunderstand me, I like cygwin very much, but whis
> that
> I see me running in great troubles.

But version skew is worse!

> At next the original kde package contains a gif6a lib which collidates with the
> gif 6b
> package of cygwin, so I decided to take the cygwin jpeg lib.
> 
> xfree for example depends only on cygwin, but kde (kfm.exe) depends from about
> 18 dll's.
> The full dependency list (a part from cygcheck -v) contains 1481 lines.
> 
> .\kfm.exe
>   C:\programme\cygwin\usr\X11R6\bin\libX11.dll
>     C:\programme\cygwin\bin\cygwin1.dll
>       C:\WINNT\System32\KERNEL32.dll
>         C:\WINNT\System32\NTDLL.DLL
>   C:\programme\cygwin\usr\X11R6\bin\libXext.dll
>   .\cygkdecore-2.dll
>     C:\programme\cygwin\usr\src\redhat\SOURCES\kde\qt-1.45\bin\qt-1-44.dll
>       C:\programme\cygwin\usr\X11R6\bin\libX11.dll
>          C:\programme\cygwin\bin\cygwin1.dll
>             C:\WINNT\System32\KERNEL32.dll
>                C:\WINNT\System32\NTDLL.DLL
>   C:\programme\cygwin\usr\X11R6\bin\libXext.dll
>   .\cygkdeui-2.dll
>   .\cygkfile-2.dll
>     .\cygkfm-2.dll
>   .\cygkhtmlw-2.dll
>     .\cygjscript-2.dll
>     .\cygkimgio-2.dll
>       .\cygjpeg6b.dll
>       .\cygpng2.dll
>         .\cygz.dll
>       .\cygtiff3.dll

But most of these are kde -- that is, *directly* under *your* control. 
The majority of the rest are X libs -- that is, always distributed
together (nobody updates libXext without also updating libX11, right?)

that leaves only cygwin itself, jpeg, png, and libz.  With the exception
of cygwin, those are pretty stable packages.

> Can you image the problems, if only one external lib will not work correctly.

Can you imagine the problems, if a user reports to ME as the
cygwin-libtiff maintainer, that she has a problem -- but I don't know
that she actually has TWO versions of cygtiff3.dll on her system?  One
that already has the bugfix but the one earlier in her path, in the
/kde/bin dir, does not (even though the API/ABI is unchanged) ?

> I remember the cygwin socket bug. I have spend one whole week with 5 hours a day
> to identifiy that cygwin must be problem, to hear that usually the client code
> has
> the problem. 

I understand and appreciate the incredible amount of work you've put
into your port.  Also the frustration when you identify a problem but
folks assume you couldn't possibly know what you're talking about, so
you have to PROVE that there REALLY was a bug.  I commisserate, really.

> So if I imaging, how much combination of problems could be, I see
> me
> supported kde 1.1.2 the whole day for the rest of my life.
> For that, I thing it is a lesser problem to provide a beta release with some
> minor
> dlls which could be removed in every next beta release.

Sure, betas -- fine by me.  But I thought you were preparing for a
"real" release -- and THAT's what got me steamed.
 
> 2) I have seen, that the xfree distribution contains a libz.dll too. Why don't
> use
> the cygwin dll "cygz.dll".
> Which zlib I should use, from the cygwin or from the x11 release ??

Wha??  We went over this about four months ago -- and the X11 release
(briefly?) began using cygz.dll.  this looks like a relapse to me. 
Suhaib?  
<<< NOTE: I will repost this subtopic in another thread >>>
 
> Again, don*t misunderstand me not, I like cygwin and the xfree server
> 
> > create a big mess down the road, and I am pretty sure maintainers of Cygwin
> > and Xfree86 will not like it.  The best way of developing and supporting a
> > package is to port your code to use the libraries from these packages
> > (Cygwin and XFree86).
> >
> Nothing else I have done. I have installed cygwin and xfree and compiled kde.
> The only
> thing I have done is to put current cygz.dll, cygjpeg6b.dll, cygpng2.dll,
> cygtiff3.dll and
> a cygncurses5.dll linked with cygwin 1.1.8 to allow users using cygwin 1.1.8
> into the
> kdesupprt package. As /usr/local/kde1/bin is by default only added to PATH
> inside kde,
> this seems to be currently no problem.

Errmmm...well, sort of.  "No problem" because those particular dll's do
not use shared memory regions or other clashable windows-isms.  That is
not always true ("never have more than 1 cygwin1.dll").
 
> Additional I have read about a "non-new-package" of cygwin, so distributing kde
> directly
> with cygwin isn*t possible at now. As I have to work on other projects in the
> next time too,
> so I have to release this beta now.

Right.  Beta.  Good.  Beta...

> I hope you understand a little bit more of my prudence. I like to say, I have no
> problem
> to work together with the cygin and cygwin-xfree guys, so i think we find a good
> way to deal
> with that all.

No problem -- sorry I got cranky.  I never saw the "beta" thing.  Now,
on to a different related subject:

> > Dependency problems
> > Some libs like jpeglib are named with the major revision number,
> > but without the minor /release or buildnumber for examples
> > cygjpeg6b.dll or ncurses5.dll.

This is GOOD.  We can't really do the whole .so-naming scheme, since (a)
every app embeds the filename of its desired DLLs, (b) the windows
runtime loader can't follow symlinks, so you'd need ACTUAL copies of:
mydll-x-y-z.dll
mydll-a-b-c.dll
mydll-i-j-k.dll
even in the case where mydll-a-b-c could replace mydll-x-y-z (but not
vice versa).  Basically, the windows runtime loader sucks. However,
libtool helps *that* problem somewhate:  in Robert Collin's current
hacked up version of libtool, it sets the dllname of --version-info
c:r:a to cygfoo-(c minus a).dll  Basically, foo.dll can replace bar.dll
if foo supports all of the interfaces of bar, but foo has additional
interfaces that bar doesn't have.  Thus, according to Gary Vaughan:

(also, see) http://www.gnu.org/software/libtool/manual.html#Versioning

> > > [libtool] should
> > > probably name [dll's] after the oldest ``interface'' (see the document
> > > quoted above) that the library fully supports.  That is, if you build
> > > a dll using libtool's --version 5:4:3, you would get library2.dll
> 
> 5:4:3 is revision 4 of the implementation of interface 5, which
> is backwards compatible with the 3 previous interface definitions
> (i.e. it is safe for applications linked against interfaces 5, 4, 3
> and 2 to load the 5:4:3 dll at runtime).   Libtool translates the
> 5:4:3 into a system specific version number for the soname to help the
> runtime loadee choose the best available library at runtime.  As I
> said before, currently this mapping is wrong on Windows, 

======== Robert's hacked libtool seems to use the "correct" mapping,
below, on windows.

> and I think
> the correct mapping is to always use the oldest supported interface
> number -- in this case library2.dll -- when generating the soname.
> This is explained fully in the version node of the libtool manual link
> that was quoted earlier in the thread. 


Anyway, for "my" dll's -- which were dll-ized before libtool even
*thought* about working with cygwin -- I just maintain the dllname by
hand.  (Of course, back when I was young and stupid, I tried to use the
package release number for that.  Libtool sez "Never try to set library
version numbers so that they correspond to the release number of your
package. This is an abuse that only fosters misunderstanding of the
purpose of library versions"

Oops.

So, libjpeg's dll name will remain "cygjpeg6b.dll" until the actual API
changes (if ever).  libreadline's name WAS "cygreadline4.dll" up thru
package version 4.1, but in the 4.2 package the ABI changed.  SO, the
new libreadline dllname is "cygreadline5.dll" -- but BOTH dll's will be
distributed for a long time to allow executables linked against
cygreadline4 to continue to work.  And libreadline's dllname won't
change again -- until the API does.

As I stated earlier, I haven't quite figured out how to handle this with
newer, libtoolized dll's, unless the (c - a) thing works correctly. 
But, since the official binutils doesn't have the appropriate support,
nor the official automake, and Robert's libtool patches haven't even
made it into *libtool's* CVS -- much less into an official cygwin
package -- we probably don't need to worry about it just yet.

> > Especially for the jepglib there is an additional runtime
> > checking if the caller has compiled in the same version lib (with
> > build-number).

Yes, that's true.  However, I don't update the  build number, unless the
API changes.  So this is a non-issue.  If it has the same name, it'll be
backwards compatible (but possibly not forward compatible; that is,
cygjpeg6b.dll(Jan-01-2001) can replace cygjpeg6b.dll(Dec-31-2000), but
(perhaps) not vice versa.

> > So when I compile kde with cygjpeg6b-2 and some time later
> > someone updates his jpeglib to Release 6b-4, kde will not be able
> > to run because of this runtime checking.

As I described, this is not going to happen.  If it does, it's a bug and
I'll fix it.

> > Because of this I decided for kde to use dlls with detailed name
> > (for example kdecore-2-0-0.dll) so that multiple installed
> > versions of one package/lib are possibly.

This is bad, since suppose I compile cervisia against kde-2-0-0.  Then,
I update kde to 2-0-1.  cervisia breaks because it can't find kde-2-0-0,
even though 2-0-1 is perfectly compatible.  I must recompile ALL of my
apps.

Under "real" unix, shared-library versions should be changed often, to
convey the maximum information to the runtime loader.  Space is not an
issue because ldconfig will fix up a bunch of symlinks.  (Plus, as much
as we scoff at the import lib/dll dualism of windows, ldconfig does the
same thing: see http://lists.debian.org/lsb-spec-9905/msg00011.html ,
esp.
 development time symlinks: libgmp1.so       libgmp2.so & libgmp.so
 runtime symlinks           libgmp.so.1      libgmp.so.2
 actual libs                libgmp.so.1.3.2  libgmp.so.2.0.2 )

On windows, you can't do this.  First, symlinks don't work.  So, suppose
we write a "winldconfig" program that analyzes all these version
numbers, and creates 82 actual COPIES of the dll's: mydll-3-2-0.dll ==
mydll-3-2-1.dll etc. Fine -- but each dll embeds its own compiled NAME. 
so, mydll-3-2-1.dll is happy, because it really is version "3-2-1" and
knows that its name is "mydll-3-2-1.dll".  But "mydll-3-2-0.dll" is
broken: the runtime loader tries to load it because "myprogram.exe"
needs "mydll-3-2-0.dll" -- but the load fails because that dll is not
REALLY mydll-3-2-0.dll" even though its FILENAME is.  In this
experiment, we've (a) written a new program (b) used up a lot of disk
space, and (c) added an install-time configuration requirement
("remember to run winldconfig") --  but we *STILL* haven't actually
solved the problem.

So, using the full libtool naming scheme is unworkable in the long run,
IMO.  But Robert's current "c - a" thing looks like a good bet.

--Chuck