[RFC]: Breaking out fontconfig and freetype into seperate packages

[Nicholas Wourms]

Harold L Hunt II wrote:

> Nicholas,
> 
> Can you explain to me what fontconfig and freetype2 are, specifically?

Freetype2, quoting the homepage [1]:
      "FreeType 2 is a software font engine that is designed to be small,
       efficient, highly customizable and portable while capable of
       producing high-quality output (glyph images)."
       ...
       "Note that FreeType2 is a font service and doesn't provide APIs
        to perform higher-level features, like text layout or graphics
        processing (e.g. colored text rendering, "hollowing", etc..).
        However, it greatly simplifies these tasks by providing a simple,
        easy to use and uniform interface to access the content of font
        files."

In other words, freetype2 provides the generic API for reading and 
preprocessing font files.  It also provides the core engine for certain 
font-related features (such as hinting, anti-aliasing, & greater 
multibyte glyph capabilities).  It currently supports:
      *TrueType fonts (and collections)
      *Adobe Type 1 fonts
      *CID-keyed Type 1 fonts
      *CFF fonts
      *OpenType fonts (both TrueType and CFF variants)
      *SFNT-based bitmap fonts
      *X11 PCF fonts
      *Windows FNT fonts
      *BDF fonts (including anti-aliased ones)
      *PFR fonts
      *Adobe Type42 fonts (limited support)

Freetype1 is an older version of freetype2 and is NOT API source or 
binary compatible with freetype2.  Some applications still depend on it 
at compile-time, but I don't think XFree86 still does.  Most notable is 
the fact that its library is libttf* as opposed to libfreetype* in 
freetype2.

Fontconfig is a library for configuring and customizing font access. It 
is the middleware between freetype2 and Xft.  But it also provides 
equivalent interfaces between freetype2 and other applications.  From 
the webpage[2]:

"Fontconfig can:

*discover new fonts when installed automatically, removing a common
  source of configuration problems.
*perform font name substitution, so that appropriate alternative fonts
  can be selected if fonts are missing.
*identify the set of fonts required to completely cover a set of
  languages.
*have GUI configuration tools built as it uses an XML-based
  configuration file (though with autodiscovery, we believe this need is
  minimized).
*efficiently and quickly find the fonts you need among the set of fonts
  you have installed, even if you have installed thousands of fonts,
  while minimizing memory usage.
*be used in concert with the X Render Extension and FreeType to
  implement high quality, anti-aliased and subpixel rendered text on a
  display.

Fontconfig does not:

*render the fonts themselves (this is left to FreeType or other
  rendering mechanisms)
*depend on the X Window System in any fashion, so that printer only
  applications do not have such dependencies."

If you are interested, the XFree86 backend (Xft2) is described in detail 
on the same page.

> Do they consist of a couple libraries and some executables?
> How many files are involved in a package that contains them?

freetype1, freetype2, & fontconfig each consist of some headers, some 
shared data, some i18n/l10n crap, and a few utility executables.

For instance, redhat has the following packages in Shrike(RH9):
[Note that the freetype1 stuff is included in the freetype2 packages]

freetype 2.1.3-6:
-----------------
/usr/lib/libfreetype.so.6
/usr/lib/libfreetype.so.6.3.2
/usr/lib/libttf.so.2
/usr/lib/libttf.so.2.3.0
/usr/share/doc/freetype-2.1.3
/usr/share/doc/freetype-2.1.3/ChangeLog
/usr/share/doc/freetype-2.1.3/README
/usr/share/doc/freetype-2.1.3/README.UNX
/usr/share/doc/freetype-2.1.3/announce
/usr/share/doc/freetype-2.1.3/docs
/usr/share/doc/freetype-2.1.3/docs/BUGS
/usr/share/doc/freetype-2.1.3/docs/BUILD
/usr/share/doc/freetype-2.1.3/docs/CHANGES
/usr/share/doc/freetype-2.1.3/docs/DEBUG.TXT
/usr/share/doc/freetype-2.1.3/docs/FTL.txt
/usr/share/doc/freetype-2.1.3/docs/GPL.txt
/usr/share/doc/freetype-2.1.3/docs/INSTALL
/usr/share/doc/freetype-2.1.3/docs/PATENTS
/usr/share/doc/freetype-2.1.3/docs/TODO
/usr/share/doc/freetype-2.1.3/docs/VERSION.DLL
/usr/share/doc/freetype-2.1.3/docs/cache.html
/usr/share/doc/freetype-2.1.3/docs/design
/usr/share/doc/freetype-2.1.3/docs/design/basic-design.png
/usr/share/doc/freetype-2.1.3/docs/design/design-1.html
/usr/share/doc/freetype-2.1.3/docs/design/design-2.html
/usr/share/doc/freetype-2.1.3/docs/design/design-3.html
/usr/share/doc/freetype-2.1.3/docs/design/design-4.html
/usr/share/doc/freetype-2.1.3/docs/design/design-5.html
/usr/share/doc/freetype-2.1.3/docs/design/design-6.html
/usr/share/doc/freetype-2.1.3/docs/design/detailed-design.png
/usr/share/doc/freetype-2.1.3/docs/design/hierarchy-example.png
/usr/share/doc/freetype-2.1.3/docs/design/index.html
/usr/share/doc/freetype-2.1.3/docs/design/library-model.png
/usr/share/doc/freetype-2.1.3/docs/design/modules.html
/usr/share/doc/freetype-2.1.3/docs/design/simple-model.png
/usr/share/doc/freetype-2.1.3/docs/freetype2
/usr/share/doc/freetype-2.1.3/docs/freetype2/build-system.html
/usr/share/doc/freetype-2.1.3/docs/freetype2/demo-programs.png
/usr/share/doc/freetype-2.1.3/docs/freetype2/drivers-list.png
/usr/share/doc/freetype-2.1.3/docs/freetype2/index.html
/usr/share/doc/freetype-2.1.3/docs/freetype2/io-frames.html
/usr/share/doc/freetype-2.1.3/docs/freetype2/library-compilation.png
/usr/share/doc/freetype-2.1.3/docs/freetype2/logo1.png
/usr/share/doc/freetype-2.1.3/docs/freetype2/platform-detection.png
/usr/share/doc/freetype-2.1.3/docs/freetype2/system-interface.html
/usr/share/doc/freetype-2.1.3/docs/freetype2/tutorial.html
/usr/share/doc/freetype-2.1.3/docs/ft2faq.html
/usr/share/doc/freetype-2.1.3/docs/glyphs
/usr/share/doc/freetype-2.1.3/docs/glyphs/Image1.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/Image2.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/Image3.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/Image4.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/bbox1.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/bbox2.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/body_comparison.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/bravo_kerned.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/bravo_unkerned.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/clipping.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/down_flow.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-1.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-2.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-3.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-4.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-5.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-6.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/glyphs-7.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/grid_1.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/index.html
/usr/share/doc/freetype-2.1.3/docs/glyphs/index2.htm
/usr/share/doc/freetype-2.1.3/docs/glyphs/points_conic.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/points_conic2.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/points_cubic.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/points_segment.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/twlewis1.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/twlewis2.png
/usr/share/doc/freetype-2.1.3/docs/glyphs/up_flow.png
/usr/share/doc/freetype-2.1.3/docs/license.txt
/usr/share/doc/freetype-2.1.3/docs/modules.txt
/usr/share/doc/freetype-2.1.3/docs/readme.vms
/usr/share/doc/freetype-2.1.3/docs/reference
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-base_interface.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-basic_types.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-bdf_fonts.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-cache_subsystem.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-computations.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-glyph_management.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-header_file_macros.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-index.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-list_processing.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-mac_specific.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-module_management.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-multiple_masters.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-outline_processing.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-pfr_fonts.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-raster.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-sfnt_names.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-sizes_management.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-system_interface.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-toc.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-truetype_tables.html
/usr/share/doc/freetype-2.1.3/docs/reference/ft2-type1_tables.html
/usr/share/doc/freetype-2.1.3/docs/tutorial
/usr/share/doc/freetype-2.1.3/docs/tutorial/metrics.png
/usr/share/doc/freetype-2.1.3/docs/tutorial/metrics2.png
/usr/share/doc/freetype-2.1.3/docs/tutorial/step1.html
/usr/share/doc/freetype-2.1.3/docs/tutorial/step2.html
/usr/share/locale/cs/LC_MESSAGES/freetype.mo
/usr/share/locale/de/LC_MESSAGES/freetype.mo
/usr/share/locale/es/LC_MESSAGES/freetype.mo
/usr/share/locale/fr/LC_MESSAGES/freetype.mo
/usr/share/locale/nl/LC_MESSAGES/freetype.mo

freetype-devel 2.1.3-6:
-----------------------
/usr/bin/freetype-config
/usr/include/freetype1
/usr/include/freetype1/freetype
/usr/include/freetype1/freetype/freetype.h
/usr/include/freetype1/freetype/fterrid.h
/usr/include/freetype1/freetype/ftnameid.h
/usr/include/freetype1/freetype/ftxcmap.h
/usr/include/freetype1/freetype/ftxerr18.h
/usr/include/freetype1/freetype/ftxgasp.h
/usr/include/freetype1/freetype/ftxgdef.h
/usr/include/freetype1/freetype/ftxgpos.h
/usr/include/freetype1/freetype/ftxgsub.h
/usr/include/freetype1/freetype/ftxkern.h
/usr/include/freetype1/freetype/ftxopen.h
/usr/include/freetype1/freetype/ftxpost.h
/usr/include/freetype1/freetype/ftxsbit.h
/usr/include/freetype1/freetype/ftxwidth.h
/usr/include/freetype2
/usr/include/freetype2/freetype
/usr/include/freetype2/freetype/cache
/usr/include/freetype2/freetype/cache/ftccache.h
/usr/include/freetype2/freetype/cache/ftccmap.h
/usr/include/freetype2/freetype/cache/ftcglyph.h
/usr/include/freetype2/freetype/cache/ftcimage.h
/usr/include/freetype2/freetype/cache/ftcmanag.h
/usr/include/freetype2/freetype/cache/ftcsbits.h
/usr/include/freetype2/freetype/cache/ftlru.h
/usr/include/freetype2/freetype/config
/usr/include/freetype2/freetype/config/ftconfig.h
/usr/include/freetype2/freetype/config/ftheader.h
/usr/include/freetype2/freetype/config/ftmodule.h
/usr/include/freetype2/freetype/config/ftoption.h
/usr/include/freetype2/freetype/config/ftstdlib.h
/usr/include/freetype2/freetype/freetype.h
/usr/include/freetype2/freetype/ftbbox.h
/usr/include/freetype2/freetype/ftbdf.h
/usr/include/freetype2/freetype/ftcache.h
/usr/include/freetype2/freetype/ftchapters.h
/usr/include/freetype2/freetype/fterrdef.h
/usr/include/freetype2/freetype/fterrors.h
/usr/include/freetype2/freetype/ftglyph.h
/usr/include/freetype2/freetype/ftgzip.h
/usr/include/freetype2/freetype/ftimage.h
/usr/include/freetype2/freetype/ftincrem.h
/usr/include/freetype2/freetype/ftlist.h
/usr/include/freetype2/freetype/ftmac.h
/usr/include/freetype2/freetype/ftmm.h
/usr/include/freetype2/freetype/ftmoderr.h
/usr/include/freetype2/freetype/ftmodule.h
/usr/include/freetype2/freetype/ftoutln.h
/usr/include/freetype2/freetype/ftpfr.h
/usr/include/freetype2/freetype/ftrender.h
/usr/include/freetype2/freetype/ftsizes.h
/usr/include/freetype2/freetype/ftsnames.h
/usr/include/freetype2/freetype/ftstroker.h
/usr/include/freetype2/freetype/ftsynth.h
/usr/include/freetype2/freetype/ftsysio.h
/usr/include/freetype2/freetype/ftsysmem.h
/usr/include/freetype2/freetype/ftsystem.h
/usr/include/freetype2/freetype/fttrigon.h
/usr/include/freetype2/freetype/fttypes.h
/usr/include/freetype2/freetype/ftxf86.h
/usr/include/freetype2/freetype/internal
/usr/include/freetype2/freetype/internal/autohint.h
/usr/include/freetype2/freetype/internal/bdftypes.h
/usr/include/freetype2/freetype/internal/cfftypes.h
/usr/include/freetype2/freetype/internal/fnttypes.h
/usr/include/freetype2/freetype/internal/ftcalc.h
/usr/include/freetype2/freetype/internal/ftcore.h
/usr/include/freetype2/freetype/internal/ftdebug.h
/usr/include/freetype2/freetype/internal/ftdriver.h
/usr/include/freetype2/freetype/internal/ftexcept.h
/usr/include/freetype2/freetype/internal/ftgloadr.h
/usr/include/freetype2/freetype/internal/fthash.h
/usr/include/freetype2/freetype/internal/ftmemory.h
/usr/include/freetype2/freetype/internal/ftobject.h
/usr/include/freetype2/freetype/internal/ftobjs.h
/usr/include/freetype2/freetype/internal/ftstream.h
/usr/include/freetype2/freetype/internal/fttrace.h
/usr/include/freetype2/freetype/internal/internal.h
/usr/include/freetype2/freetype/internal/pcftypes.h
/usr/include/freetype2/freetype/internal/pfr.h
/usr/include/freetype2/freetype/internal/psaux.h
/usr/include/freetype2/freetype/internal/pshints.h
/usr/include/freetype2/freetype/internal/psnames.h
/usr/include/freetype2/freetype/internal/sfnt.h
/usr/include/freetype2/freetype/internal/t1types.h
/usr/include/freetype2/freetype/internal/t42types.h
/usr/include/freetype2/freetype/internal/tttypes.h
/usr/include/freetype2/freetype/t1tables.h
/usr/include/freetype2/freetype/ttnameid.h
/usr/include/freetype2/freetype/tttables.h
/usr/include/freetype2/freetype/tttags.h
/usr/include/ft2build.h
/usr/lib/libfreetype.a
/usr/lib/libfreetype.la
/usr/lib/libfreetype.so
/usr/lib/libttf.a
/usr/lib/libttf.la
/usr/lib/libttf.so
/usr/share/aclocal/freetype2.m4

freetype-utils 2.1.3-6:
-----------------------
/usr/bin/fterror
/usr/bin/ftmetric
/usr/bin/ftsbit
/usr/bin/ftstrpnm

freetype-demos 2.1.3-6:
-----------------------
/usr/bin/ftdump
/usr/bin/ftlint
/usr/bin/ftmemchk
/usr/bin/ftmulti
/usr/bin/ftstring
/usr/bin/fttimer
/usr/bin/ftview

fontconfig 2.1-9:
-----------------
/etc/fonts
/etc/fonts/fonts.conf
/etc/fonts/fonts.dtd
/usr/bin/fc-cache
/usr/bin/fc-list
/usr/lib/libfontconfig.so.1
/usr/lib/libfontconfig.so.1.0
/usr/share/doc/fontconfig-2.1
/usr/share/doc/fontconfig-2.1/AUTHORS
/usr/share/doc/fontconfig-2.1/COPYING
/usr/share/doc/fontconfig-2.1/README
/usr/share/man/man3/fontconfig.3.gz

fontconfig-devel 2.1-9:
-----------------------
/usr/bin/fontconfig-config
/usr/include/fontconfig
/usr/include/fontconfig/fcfreetype.h
/usr/include/fontconfig/fcprivate.h
/usr/include/fontconfig/fontconfig.h
/usr/lib/libfontconfig.so
/usr/lib/pkgconfig
/usr/lib/pkgconfig/fontconfig.pc

However, as a disciple of the packaging gospel according to Charles 
Wilson, I will be sticking to the system he uses.  This is how I've 
tentatively split the packages:

freetype1         - contains core exe's and/or scripts + runtime shared
                     data + manpages + documentation
freetype1-contrib - contains extra/non-essential exe's and/or scripts
freetype1-devel   - contains headers + static&import&libtool archives
                     + *-config scripts + pkgconfig data
libttf2           - contains the single runtime dll
-------------------
freetype2         - contains core exe's and/or scripts + runtime shared
                     data + manpages + documentation
freetype2-contrib - contains extra/non-essential exe's and/or scripts
freetype2-devel   - contains headers + static&import&libtool archives
                     + *-config scripts + pkgconfig data
libfreetype(n)    - contains the single runtime dll (n being the libtool
                     versioned dll identifier)
-------------------
fontconfig        - contains core exe's and/or scripts + runtime shared
                     data + manpages + documentation
fontconfig-devel  - contains headers + static&import&libtool archives
                     + *-config scripts + pkgconfig data
libfontconfig(n)  - contains the single runtime dll (n being the libtool
                     versioned dll identifier)

In addition, I plan to include ttfmkdir, which assists in generating 
font config files specifically for TTF's.

> Are any of their libraries linked to from within the standard XFree86 
> distribution?

Yes, but the sources are actually contained in xc/extra/freetype2 and 
xc/extra/fontconfig and there are Imake directives to automagically 
handle external versions as opposed to internal versions.  This is how 
the aforementioned vendors do it, by adding the following directives to 
host.def (or whatever you use for xcompiling)*:

HasFreetype2  YES
HasFontconfig YES
HasExpat      YES
HasZlib       YES

The Imake system will look in the standard location under /usr for these 
libs, so no additional fuss is necessary.  It should be fairly automated 
from there.  Do make sure to include these directives in the default 
configuration when installed.

I also plan to keep abreast of any vendor and/or XFree86 patches to 
these libs, so there ought to be no negative impact on how XFree86 behaves.

*Note that I haven't inspected your tree to see what directives you are 
using, so I might be redundant in some cases.

> The reason I ask these questions is that I am all for splitting out 
> fontconfig and freetype2, as long as I can cleanly exclude them from the 
> packaging scripts and if I don't have to do too much recompiling when 
> new fontconfig/freetype2 packages are released.

No problem, I understand.  I anticipate that the installation size will 
not or only slightly exceed the size for current dependencies.
I anticipate that the only packages actually required by the core server 
should be:
*libfreetype(n)
*libfontconfig(n)

If any fonts are installed, then we add to the dependencies (this is 
because the post-install scripts for the fonts may need to run some of 
the utilities):
*freetype2
*fontconfig
*ttmkfdir(possibly)

Finally, I see that the following being dependencies of the XFree86 
devel package:
*freetype2-devel
*fontconfig-devel

The way the scheme works, even *if* they make horrendous binary 
incompatible changes to the core libs (thus changing the result of the 
libtool versioning expression [[CURRENT] - [AGE]]), you'll still be 
fine.  Since the gospel of packaging according to Chuck dictates that 
prior runtime library versions must still be available, you need not 
recompile against the new version until YOU are ready to.  Think 
ncurses.  Of course, if some incompatibility does slip in without a 
versioning bump, I can always manually bump it.

That being said, you'll initially have to recompile against the new 
libraries since libtool versioning is slightly more sophisticated then 
Imake versioning (unfortunately), thus meaning the runtime dlls are 
probably going to have different numerical suffixes.

Also, a few vendor patches may need to be applied to the core sources 
now and in the future if there are critical issues with font 
readability.  Of course, this is at your discretion and definitely at 
your convience as to when and if you do it.  I guess we can cross that 
bridge when we come to it.

Finally, we might want to dll'ize some X libraries which are currently 
only built static.  I hope (and pray) that we can nip the libXt problem 
in order to get a shared libXt.  This is currently what prevents me from 
releasing a new Xaw3d with working shared libs (static is ok).  Still, I 
don't anticipate the libXt problem to enter into the scenario I'm 
proposing now.  Also, I noticed that we could probably change some of 
the defines when we migrated to 1.5.0, most notably is that the alloca 
interface is now provided by cygwin and cygwin has much more support for 
widechars.

Ditch -DNO_ALLOCA and replace with -DINCLUDE_ALLOCA_H for 
AllocateLocalDefines.

Ditch "-DUSE_XWCHAR_STRING -DUSE_XMBTOWC" and use "-DHAS_WCHAR_H 
-DHAS_WCTYPE_H -DHAS_ISW_FUNCS -DNO_WIDEC_H" for XawI18nDefines.

I'd willing to do some careful auditing of our current defines to see 
what else the cygwin dll is providing that we aren't taking advantage 
of.  Of course, I'll leave that for future discussions.

> Answer those questions and we can think about it.
> 

I hope my response was detailed and concise enough to alleviate any 
fears.  I look forward to further discussion.

The good news is, that by using the most current of these libraries, 
most newer versions of our included windowmanagers will support using 
the truetype fonts found on every Windows box.  Openbox, WindowMaker, 
and lesstif all have some base support for these libs.  Unfortunately, 
the older fontconfig distributed with 4.3.0.1 had some pretty notorious 
issues, but it is too long of a story to get into about why XFree86 is 
so slow to update (This centers around the whole Keith 
Packard/XFree86-core falling out).

Cheers,
Nicholas

[1] http://freetype.sourceforge.net/freetype2/index.html#features
[2] http://fontconfig.org/