Making desktop Linux software installation easy 28th September 2004 Isn't it easy already? No ... technologies like apt have serious drawbacks we should recognise: * Centralisation introduces lag between upstream releases and actually being able to install them, sometimes measured in months or years! * Packaging as separate from development tends to introduce obscure bugs caused by packagers not always fully understanding what it is they're packaging. It makes no more sense than UI design or artwork being done by the distribution * Distro developers end up duplicating effort on a massive scale. 20 distros == the same software packaged 20 times == 20x the chance a user will receive a buggy package. Broken packages are not rare: see Wine, which has a huge number of incorrect packages in circulation, Mono which suffers from undesired distro packaging, etc. * apt et al require extremely well controlled repos, otherwise they can get confused and ask users to provide solutions manually : this requires an understanding of the technology we can't expect users to have. * Very hard to avoid the "shopping mall" type user interface, at which point choice becomes unmanagably large: see Synaptic for a pathological example of this. Better UIs are possible but you're covering up a programmatic model which doesn't match the user model esp for migrants. * Pushes the "appliance" line of thinking, where a distro is not a platform on which third parties can build with a strong commitment to stability but merely an appliance: a collection of bits that happen to work together but may change in any way tomorrow: you can use what's on the CDs but extend or modify it and you void the warranty. Appliance distros have their place: live demo CDs, router distros, maybe even server distros, but not desktops. To compete with Windows for mindshare and acceptance we must be a platform. Apt does have its place: useful for servers, core OS packages (see below) We want to support a variety of UI paradigms: MacOS X style drag/drop, setup.exe, apt-get install foo - all of these are possible in a distributed world IMPLIES: - A way for maintainers to build easy to use universal binary packages in a decentralised fashion (but which can still perform dependency resolution) IN PROGRESS: autopackage, ZeroInstall ALREADY EXISTS: Loki Setup, BitRock etc (no dep resolution) - A unified platform to reduce the strain on the dependency resolvers and increase reliability consistent/coherant API policy, lack of API duplication, centralised management not required, win32 shows us that a platform doesn't have to be coherent to be widely used basically means a large set of functionality that you can get with a single expressed dependency. Either your distro provides these packages or it doesn't, in which case you have to do dep resolution manually. Backwards compatibility is managed, ie if you depend on Platform v1.0 then Platform v1.2 will satisfy this. Can be expressed via virtual dependencies in a repository using todays technology (apt et al) so does not require distro co-operation. NOT STARTED: lsb does not meet above requirements of size, informal backwards compatibility efforts etc. - Sucking bulk of distribution policy upstream, eg init scripts, config files (debian .d dirs) rather than having it expressed via proprietary packaging policy. This is already underway, eg with freedesktop.org menus replacing each distros/desktops customized menu scheme. Currently not widely implemented, Fedora forked Gnome to add it but upstream has more stringent requirements - no action on this as of writing. Long term will (hopefully) replace vFolders, Debian/Mandrake Menu System etc. Need forum like xdg-list but for distro developers rather than desktop developers where policy can be standardized: currently does not exist, no plans to create one. NOT STARTED: no distribution developer forum exists - Desktop integration via a dedicated PAL (packaging abstraction layer): allows us to drive good package management UI deep into desktop without compromising portability: despite different implementations primitive operations mostly identical therefore ripe for abstraction DECENTRALIZED BINARY PACKAGING - Examples in use today: Loki installers, self extracting archives, distro neutral RPMs (eg, Sun JRE). Typically lack dependency resolution but not needed due to highly defensive coding style (lots of dlopen action, no assumptions about base system) - Future possibilities: autopackage, Zero Install, LSB RPMs - autopackage is interesting because it provides a good user experience right out of the box: people using unmaintained alpha autopackages of Gaim from over a year ago because "it just worked" and "I couldn't figure out how to install it any other way". => Many Linux users without much technical ability, major change from the past => the "auto" in autopackage is important to people However, InstallShield type UI *not* long term goal. Complex solution to a messy world. - ZeroInstall is interesting as it has a MacOS/RISCOS style appfolder UI system working today, and because it's conceptually clean. However, requires installation of a kernel module (so initial install can be difficult) and does not integrate so well with existing distributions. - These two have another advantage: they are not native to any pre-existing distribution so can be hated equally by all - Binary portability not so simple, technique used today is build on RH6.2 box and pray, technique used by autopackage/ZeroInstall is apbuild. apbuild is "policy in a box", will evolve and adapt over time to match peoples needs. Essentially about altering upstream decisions. - Big issue is library maintainers not following best practices like symbol versioning, header versioning, parallel installability - Key libs break backcompat too often (eg, OpenSSL) - Meets resistance from those who believe there's nothing wrong with the old way. eg, "project policy is that we don't provide binaries", "compiling software is more pure, it's the open source way", "it's the job of the distribution to provide binaries" etc etc - Plan is not to replace apt and distro packaging but to complement them, distro packaging becomes only for "core" software. Good interop between neutral and native solutions needed. What is core? UNIFIED BASE SET: - Could be an upstream project like any other, or could be freedesktop style informal agreements. - Most distros already package most stuff that'd be in such a base set anyway, so key challenges are to: - manage versioning (ensure *-compat packages are always available) - ensure it's easy to install the base packages, ie by providing package repositories and virtual dependencies - ideally get it into the distro "base sets" where that makes sense (fedora, mandrake, suse ...) - convince people it's a good thing to do! - Linux Standard Base: - Different approach, based on specifications of libraries rather than specifying individual package versions. Reasoning: allows forking, blocks distro patching which modifies API/ABI. - Good: * Formal standard people can write test suites for * Support from large [proprietary] ISVs - Bad: * Not realistic to specify ABI for every library we need in unified base set to be competitive, doesn't achieve much anyway once you move beyond the lower levels, ABI spec only a subset of what's needed * Moves too slowly * Standardises existing practice, ie is not able to stop NPTL style breakage * Difficulties w.r.t conflicting goals between LSB and upstream, for instance C++ ABI vs ISO * Blocker: experience shows that real world software developers don't write to specs, they write to implementations: very few examples where this is not the case. Therefore attempt to specify eg GTK+ ABI doomed to fail w.r.t backcompat in long run: a) Too much behaviour unspecified: callback ordering, handling of error conditions, sorting etc b) People end up depending on behaviour accidentally anyway: construct only GObject properties - People don't always deserve to lose because they wrote buggy code: - Often docs are unclear or exact behaviour is unspecified. GObject properties case: no official specs at all - No other way: how many people write their code to be independent of callback ordering/multiple callbacks? - Everyone writes buggy code. People are expected to support each other even when they screw up. - Ultimately it's the user who loses - Potential solution? A mostly informal, flexible base set maintained by a standard heirarchical team rather than a committee who are willing to patch upstream binaries in order to maintain compatibility with pre-existing code: the Microsoft approach - Not as extreme, majority of software on free desktop free software, can be upgraded for zero dollars (automatically?) even if it's a pain for the user, so affects breaking change cost/benefit analysis - However, basic theory is the same: don't break compatibility, this is more than ABI/API stability and includes things like buggy behaviour apps depend on. - We want to support games, these are almost always proprietary and worse they have a max shelflife of a few months: vendors often not interested in fixing bugs after this date, even if people are still playing years after release - Advantage: can be fast moving, can do things a formal organization would frown upon, can JUST DO IT - Disadvantage: lacks legitimacy - Packagers "opt in" to shared base set by setting their only dependency as "base-set > 1.0", by adding rpaths to their binary ${ORIGIN}/../lib/base-set to allow for upstream overrides - Only applies to userspace, kernel is fundamentally screwed from an stability (== ease of use) perspective *however* as kernel becomes ever more mature it becomes more practical to simply fork a stable series and backport non-breaking changes/driver upgrades. Kernel is mature, biggest reason to upgrade is improved hardware support. - Alternative: attempt to abstract subset of unstable kernel ABI behind a stable one provided by a module. Fighting a losing battle? - Software is not art, hacks OK if they serve the user and don't compromise future engineering. - Only keep driver compat in stable series SUCKING DISTRO POLICY UPSTREAM - Need equivalent of xdg-list for distributions, need more manpower devoted to actually implementing and ratifying standards, eg recent files spec, menu spec mostly in limbo. - A standard that is only implemented "sometimes" or "occasionally" is just as bad as no standard at all, therefore implementability crucial in design phase otherwise you end up with the menus problem. - Examples of distribution policy: - Dealing with upstream goofs, eg libpng parallel installability, glibc non-releases - Extensions to standard config files, /etc/modules - Co-ordinating potentially breaking changes like NPTL - Obvious stuff: init scripts, shared config ui (overlap with xdg), maybe even co-ordinated branding. - new freedesktop list appropriate forum, or new project entirely? - Can be hard to get distros involved, often distro developers out of sight/not active in community. Let's get Havoc to do it! :) -------------------------------- PROBLEM ---------------------------------- - If versioning policy is moved upstream, and packaging decentralised WHAT IS A DISTRIBUTION? - Some distros define themselves through their packaging policy, others through their desktop, others through their support (mix'n'match to your pleasure), others are highly specialised (eg knoppix live CD, router CDs, embedded distros etc) - focus is on standardising platform not higher levels: nobody cares if you prefer compiling your own X server, or use a different icon theme as long as it's compatible with everybody elses and can be depended on via a shared base set - will meet resistance from those who believe their preferred distro is the one true way: "why do people complain, just use Debian" mentality. - we all have more interesting things to do that reinvent the distribution wheel over and over : "wouldn't you rather be hacking anyway?" DESKTOP INTEGRATION - Desktop projects are portable, therefore requires an abstraction that allows them to maintain that portability. Do what HAL is doing for hardware, but for packaging: PAL! - 'Ideal' UI varies between users: drag/drop appfolders, apt-get install foo, setup.exe, different paradigms but all supportable if we're smart - drag'n'drop a la MacOS X/RISC OS except that actually works - MacOS X not consistent, Apples own software doesn't use appfolder installs, custom installer program shipped with own OS and even then developers roll their own. - RISC OS/NeXT never had to survive in the internet age of 100s of programs installed at once, exponential OS complexity/integration etc. - How? Extend the .desktop abstraction to include packaging data, such that a desktop can take an arbitrary application .desktop file and install the package it "points" to behind the scenes, with no user interaction required. Therefore drag from webpage to panel == download+install operation. Can click icon in web page directly to launch it (or from CD, or email, or IM conversation). - Requires standards for packaging metadata, see PAL - Not hard, requires several months of effort from the right people, requires getting desktop projects on board (ie, backwards compatibility with existing systems like apt). - apt UI can still work in a distributed world: DNS type hierarchy for software. - Possibility for insertion of whitelisting, attack malware by giving user warning if they try and install it (optionally block entirely) - Can be done in parallel with other tasks. ****************************************************************** * None of this is technically difficult, it just * * requires a change in the open source culture. * ******************************************************************