Monthly Archives: May 2009

Karmic Desktop UDS run-down!

I just got back from a wonderful UDS in beautiful Barcelona and thought I would provide a summary of what we can expect in the Karmic Koala 9.10 desktop. Keep in mind that I don’t speak for Canonical and what follows is just my understanding of what is on the table for Karmic.

Overall it is gearing up to be a pretty radical and exciting release; there are some changes to the default application set as well as some major version upgrades of existing core components. We are trying to be fairly aggressive in terms of new stuff so that if Canonical wants Karmic+1 to be an LTS (Long Term Support) release, we can have fairly stabilized new technologies by then (thanks to 6 months of stabilization in the Karmic cycle) instead of having to wait until after the LTS (Karmic+2) to introduce them. Since many of these changes would be too radical to first appear in an LTS, if we don’t upgrade now we may not be able to for a year, and have to maintain old versions for 3-5 years in the LTS.

On the messaging front, Ekiga will be dropped from the CD to save 10 megs, and Empathy will likely replace Pidgin due to a responsive upstream, voice/video support, and better GNOME integration. It also now has the ability to import accounts from Pidgin, so this should help out with migration. I checked it out a bit at UDS and was impressed with how useful it is with absolutely zero configuration. It will pull your name from the system and enable avahi (auto-discovery of people nearby, like bonjour) with no set up, which made it quite easy to get in contact with people at the conference. You can also supply your email and Jabber ID to the avahi interface to allow other people to discover that info as well.

It also sounds likely that Banshee will replace Rhythmbox as the default media player, and it is the official default of UNR (Ubuntu Netbook Remix) Karmic. This will bring a snazzier interface, better device support including iPods and Androids, and quite importantly an active and responsive upstream. I will admit to not being a huge fan of this transition for Karmic as it seemed too early for me (the lack of a folder watch is quite a regression for me, and it has been promised for the last 3 releases or so, so I’m not holding my breath), but after checking out 1.5 for a bit I will admit that it is growing on me. The user interface does seem nicer, and the lightweight video library which keeps track of what you haven’t watched is nice. However, it does seem to use 3-10x more memory than RB which is very troubling (60-300MB compared to RBs fairly consistent 25MB), especially on the netbook scene. I’ve also had issues with it skipping occasionally, which is very unfortunate. Hopefully the UNR switch will put pressure on better memory management for Banshee.

Banshee syncing with an Android G1

Empathy and Banshee will probably replace their predecessors around alpha 2 of Karmic, and will be either left as default or reverted based on reported regressions and bugs. Keep in mind that if you end up not preferring these applications, the other ones still exist and you can continue to use them.

There are also going to be a bunch of underlying speed improvements, with the boot speed goal being 10-12 seconds. When Ubuntu talks about boot times, we are referring to the time from when grub starts (when Linux first gets control of the machine) to when the user is at a fully loaded desktop with no I/O. The main test machines being used by Canonical here are Dell Mini 9s, with auto-login enabled to get a consistent log-in time. This is pretty impressive as the boot goal was 25 seconds in Jaunty, which was met, and was aggressive itself as Intrepid booted in about 65 seconds on the Dell Mini 9.

grub2 is likely to be default for new installations (upgrades will have grub1 chainloaded to grub2), with ext4 as the default filesystem. The boot process will also be streamlined, with the grub timeout set to 0 and the boot menu hidden. There will instead be two new ways to boot into a different system now. First, there will be a key that can be pressed while booting to bring up an OS chooser, which will halt the current boot and restart into the chosen one. Another goal is to have the restart menu item in GNOME aware of installed OSes and allow the choice there, so you could select for example “Restart into OSX”. All in all this means no racing to select the OS for dual-booters, and a faster boot process as well. /tmp is also hopefully going to be made a tmpfs, which means it will reside in RAM and overflow to a swapfile (which in recent Linuxes have on par performance with swap partitions). This means power savings, less disk I/O (especially great for SSDs), and of course blazingly fast performance which should help out a lot especially when, say, loading files from inside an archive The Gnome Display Manager (GDM, which handles the login screen) will also likely be upgraded to GDM2.

Finally let me fire off a few more changes. Power management is being improved all around, with one change already landed being that audio cards will be automatically powered down after 10 seconds of no sound. Encrypted Home directories will hopefully be easier to set up now with an option right in the graphical installer, and I’m working on a UI for managing this and encrypted Private directories in Karmic, more on that later. Firefox 3.5 should be the default version of Firefox. For notifications which want to display actions if the user is interested, there is work on morphing windows:


Ubuntu is also working on being social from the start (see desktop-karmic-social-from-the-start on gobby.ubuntu.com), perhaps installing Gwibber by default and asking the user if they want to integrate social sites (twitter, facebook) into the desktop when they visit them in Firefox, via an extension. There has also been work in looking for a better scanning application to replace xsane (perhaps GnomeScan), some look into using Gnome Control Center, and a common printing dialog.

Okay phew, that’s what I’ve got to report! Let me know what you think of these decisions and changes, and if there is anything you were hoping for that didn’t make it, or really anything else you’ve got to say!

Comments

News Blog
Nice Post
Steven Spurrier
Michael
Regarding the last anonymous post, I'm not even sure if you read my post. KMS support IS planned for the next release, and I specifically mentioned it. Delta updates were also discussed at the summit as well.

Since Ubuntu and Fedora have different release schedules, they are going to have differences as different technologies and features mature at different times. Both distributions have had certain features before the other and that is surely the way it will continue.
Anonymous
what a boring release it will be…
app1 changed to app2…
ubuntu is really getting years behind fedora:
- no multiseat (MDM) support
- no delta updates
- no smooth loading (KMS) support
and it's not even planned… dispite the fact that some of these features were requested for years!
Anonymous
"Ekiga will be dropped from the CD to save 10 megs"

why not save lots of megs by leaving out mono/banshee/tomboy?

there's gnote to replace tomboy, and banshee doesnt appear to be fully functional..
Anonymous
Banshee? Mono fail!

Empathy has no UI for dialing out on voice calls, doesn't have metacontacts, and its SIP and IRC support are underdeveloped. I've been tracking development and testing it for years. It's still crap for usability, though I understand it's an abstraction-obsessed programmer's wet dream.
The Mad Hatter
We don't use VLC for legal reasons. The same reasons apply to Mono applications. Mono needs to be purged from Ubuntu, along with the applications that use it (FSpot, Tomboy, Banshee). This will also save a considerable amount of disk space.

I would also suggest changing out Evolution for Thunderbird. The amount of integration between Evolution and the Gnome Desktop environment reminds me to much of the way Internet Exploder is integrated into Windows, and the problems caused by that.

As a final note, I think that we should consider not using any software which isn't GPL/BSD/MIT/Apache/Mozilla licensed. License proliferation is becoming a serious issue, which can lock code from reuse. The above licenses are (at least in my opinion) the best ones from a point of sharing.tr
Anonymous
Agreed with the above poster not thrilled with Banshee on two counts: 1) it is not nearly as lean as RB, which is quite a snazzy app in of itself, and 2) it is encumbered by .NET and Mono infested. I think it would be good for the distro’s to heed R. Stallman’s advice and stick to GNU applications, and if closed source must be included make sure its from a originator without a history of exploits of such.
Blimundus
Thanks for the run-down! These are very interesting developments to hear about.

I am particularly interested in the debate on the scanning application.

Etienne’s work on GnomeScan started off with the right design choices, and with the goal to make scanning as ubiquitous as printing. In other words, as I understand it, GnomeScan is all about integration with other programs, the file chooser etc… I think it would be great to see this becoming a reality.

GnomeScan would benefit from Gnome/Ubuntu giving it some attention though.
Bayger
Michael: I have added a blueprint for the idea suggested by me. It’s called desktop-karmic-color-themes. I don’t know what to do next with this. This is my first blueprint. I don’t even know if this is the usual place to post such ideas.
Jeffrey Seguerra
I read lot of rants about mono… was mono discussed in Karmic UDS?

http://tinyurl.com/pycr9u
Jef Spaleta
You had sessions at UDS about gnome 3.0…but is Canonical talking directly with upstream about integrating Canonical built technologies like libindicate?

UDS is a walled garden. Don’t misintepret talking “amongst yourselves” as a substitute for discussion with upstream.

Packagekit adoption would be an example. packagekit-gnome is a gnome module now. It’s moving along a path towards deep integration into the GNOME desktop. How much work has Canonical invested release after release into refining its distro-specific update manager experience? How much of that manpower expenditure work is directly working against adoption of PackageKit as a common cross-desktop solution?

PackageKit adoption has originally brought up for intrepid UDS. I’m not holding my breath for Karmic. Especially in light of the AppCenter proposal which is an even more aggressive divergence path than the current gnome-app-install tool.
https://wiki.ubuntu.com/AppCenter
Michael
Alexander, I think there was a PackageKit session, I’d check summit.ubuntu.com and look at the schedules. If there is one it should be linked to a blueprint / wiki page.

Jef, what you are pointing out is definitely true. If we diverge too much and upstream stops maintaining what we are patching, we are either left maintaining it ourselves or porting it, either of which is extra manpower as you say and is suboptimal. We are looking long term and there were some sessions at UDS on GNOME 3.0, Shell, Zeitgeist, Control Center and such. The plan is to have as many GNOME 3.0 components as possible easily installable in Karmic, and solicit some early feedback on user experience, regressions, and integration with other Ubuntu aspects.

There is definitely some exciting work going on to make it really easy to use the latest upstream versions of packages in future Ubuntus, which should really help upstreams get better feedback from us and make development upstream easier as well. Plus it should eliminate some surprises by pointing out integration issues early.
Jef Spaleta
Do you know if upstream gdm has incorporated the presence functionality Canonical introduced in Intrepid? Is Canonical going to forward port that or is essentially duplicating functionality in the gdm trunk with a custom implementation?

The differentiated guest and presence features that Canonical developed against the 2.20 codebase was one of the reasons holding back from using gnome 2.26 in Jaunty. And it appears to be something Canonical is going to continue to spend manpower on.

You have to be careful, the more you differentiate and rely on those differentiated features, the harder it is for you to continue to move forward as upstrem executes its own feature roadmap. The fact that Canonical is having to spend time refactoring its customized guest session and customized fast-user-session code..code that was created against the older 2.20 gdm codebase..means Canonical has less manpower resources to help finish off the gdm rewrite.

And looking even longer term, how much of the differentiated work that Canonical is doing across the Gnome desktop more generally is going to make it into the vision for Gnome 3.0? Is all the effort that continues to go into making libindicate and notify-osd work across the application space going to be incorporated into what gnome 3.0 is? Will it even fit in with gnome 3.0 concept of a shell interface? How much is Canonical engaged in upstream roadmapping for new functionality? You can’t really argue an “agree to disagree” attitude if Canonical isn’t taking part in the roadmapping discussion and making the case for integration of their technology implementations.

-jef
Anonymous
kubuntu has packagekit now, so i imagine there is at least SOME work being put into it
Alexander
Any news on integrating PackageKit as the principal software installation mechanism?
Michael
Thanks for reminding me of that feathertail. I didn’t follow that too much but the blueprint is here: https://blueprints.launchpad.net/ubuntu/+spec/desktop-karmic-integrating-with-ubuntu-one . The plan is to make it as easy to use as possible (and potentially integrate into migration-assistant if you already have it) without being invasive such as popping anything up or adding an unconditional step to the installer. The client is not proprietary, it is open-source and in fact I have already used the source to help me in another project so I am quite grateful for it!

Jef, that is a great question. I am not an authority on this topic (you might try dbarth and seb128?) but my understanding is we want the bleeding edge of GDM, so definitely 2.26, I assume 2.28. Since that is the case it should be easy to upstream, but whether we are doing the work upstream or downstream I don’t know. Upstream is the preferred approach so I imagine if upstream wants this feature, we would. Ubuntu gets knocked for doing too much work downstream and while some of this is deserved (there was in fact a general session on this), in some cases upstream is not interested in our direction, and that is the purpose of being a different distro after all. If we weren’t doing anything different there wouldn’t be a point to existing!
Jef Spaleta
What version of gdm is Canonical committed to shipping in Karmic?

Jaunty is still shipping the gdm 2.20 release while the last upstream release was gdm 2.26.
http://ftp.acc.umu.se/pub/GNOME/sources/gdm/

Is the restart option for gdm going to be developed against gdm upstream trunk in collaboration with upstream gdm? Or is it going to be a downstream patch built against the old gdm that Ubuntu ships like some of the previous gdm patchset that Canonical developed in the Intrepid development time period?
feathertail
What about the Ubuntu One integration that I have been hearing about? And if it’s being included, isn’t that kind of distressing when it’s proprietary?
Michael
Anonymous, upstream GNOME isn’t switching to Banshee as far as I know (but Banshee would eventually like them to) but as andreasn points out below (thanks!), apparently GNOME doesn’t ship one at all except totem. So this isn’t a divergence from upstream, just a change on the Ubuntu side. And in fact going to Empathy is in fact a convergence bringing us closer to GNOME! As far as upstreaming the boot stuff, last I heard (last UDS) Debian was not interested due to how wide-reaching the changes are, but maybe they’ll come around. This also could have changed recently.

nailor, you can find the blueprint at https://blueprints.launchpad.net/ubuntu/+spec/desktop-karmic-gnomescan. I’ll add this to the post.

gilir, I agree and that was one of the first things I noticed as well that was disappointing about Banshee; no fade in/out.

Alex, I think Mono is more modern and should allow easier extensibility and maintenance as well as make new contributors more likely. Plus tomboy and gnome-do are already using it. As far as PackageKit goes, I’m not sure, but you can check out summit.ubuntu.com for the full schedule!

Mackenzie, thanks for the info and I have heard that, but I doubt that a custom widget takes THAT much more memory; something else is definitely going on. Plus, my library is about 50 gigs and 8,000 songs, so I feel like I am on the upper end of the average desktop user anyway. I don’t think we should be harming the experience of people with libraries of this size, so someone can have 220GB of music with less problems :) I’ve also had issues with it skipping on my Core 2 Duo, which RB never does, so that is unfortunate. That said, I do think Banshee is headed in a cool direction and I’m excited to see where it goes.

Bayger, I don’t know of that being considered but it could be cool; I’d personally love for a “look and feel” choice in the installer which lets you enable a dock instead of a bottom gnome-panel as well.

Pro, the idea with boot choice is to make it fairly simple (I think pressing something like shift/alt/ctrl) and make it as discoverable as possible, but with Windows or OSX you just “have to know” so we don’t have much to go by as far as presenting this. Also, the restart option should make it fairly easy.

Pēteris, I’m up in the air about some of the changes as well, but we’ll see how feedback goes. If upstream is as responsive and interested as claimed, the kinks should be worked out. And indeed, gnomescan does seem to be what is being considered.
Pēteris Krišjānis
Pidgin replacement - good, RB replacement - bad, for scanning - Gnome Scan please. Everything else sound good :)
Pro
The new boot menu which makes grub time set to 0 might cause a headache for new users to ubuntu who are just trying it and would like to have easy access to their win/osx os also.
it seems this step needs to be executed really well for it to be good for both experienced and new uwsers.
Anonymous
Great news indeed! Thanks for your report.
It seems that Karmic will be awesome.

(Even if I don’t feel completely trustworthy Banshee for Mono…)

@Bayger
Usually I keep the brown colour but I like your idea.
andreasn
Nice to hear Banshee is moving closer to getting included. The player have a really good and slick UI, and hopefully it means we can convert more people to Ubuntu.

Anonymous: GNOME upstream don’t have a default music player included in the desktop module set [1] (unless you count Totem)

1. http://live.gnome.org/TwoPointTwentyfive/Desktop
Bayger
Great news! By the way, have you considered adding desktop theme selection during installation of Ubuntu 9.10? I am sure that a lot of people would be very grateful to have a choice between standard brownish theme and 2 or 3 other colour schemes (green, blue). This would be great option, especially for beginers. And yes, people are a little bored of all those browns everywhere.
Mackenzie
Banshee has a higher memory overhead as a baseline, but it does not require more memory for larger libraries the way Rhythmbox does. This is due to a Banshee-only widget which only needs to store the visible items in the song/album/artist list plus 10 above and 10 below. Rhythmbox’s stock GTK list viewer must hold ALL items, so if you have a very large library, it could easily use hundreds of megs of RAM.

If RB consistently uses 25MB for you, you must have a small library. For people like my friend with 219GB of music, RB uses much, much more.
Alex Eftimie
I don’t like the idea of Banshee either, mostly because of .NET implications.

Anything else sounds great!

How about PackageKit integration? Have you discussed that?
gilir
Banshee lacks also the gapless/crossfading playback, which is a NO-GO for me. Sure Banshee have cool features, but it should not hide the problems Banshee have (memory, crash, slow …).
Also RB is not so inactive, the 0.12.2 was released some days ago and there are 2 GSOC on RB.
nailor
Michael, can you provide a pointer to more info about the ongoing investigation of scanning applications?
Anonymous
Is GNOME upstream switching to banshee too?

I hope you push those speed improvements back to Debian.

A teaser: Desktop integration with encrypted directories for Karmic

Recently I’ve been working on desktop integration with ecryptfs. Dustin Kirkland has done some awesome work enabling encrypted Private directories, as well as entirely encrypted Home directories, and I want to bring a UI to that goodness for the Karmic desktop.

UbuntuOne displays a banner at the top of its shares, and this inspired me to borrow the code for use with encrypted Private directories. After a bunch of hacking and debugging, I finally got something to show up:


Pretty exciting! There is much work to be done behind the scenes but this is an encouraging start. After I get this working I plan on making a UI for installing ecryptfs-utils (the necessary package), setting up an encrypted Private directory, and managing/configuring one (or an encrypted Home). This UI would perhaps be available from System -> Administration -> Encrypted Directories, and would allow a user to have a directory of encrypted files available in a few clicks, which is mounted/unmounted transparently at login/logout.

What do you think? Are you currently using an encrypted Home or Private directory? Would you be more likely to if there was a UI to set it up? Please share your thoughts and comments :) I’ll be at UDS and can schedule a session on this if there is interest, as well.

Comments

Perpetual
Hi, I really need something like this. My family multimedia center, based on ubuntu, contains a lot of movies (horror and such) that my small children better not click on by accident. Still, I want them to be able to access the cartoons section in the Movies directory. Everybody just uses my account, it is always logged in, so decrypt at login does not cut it. Instead I need something that pops up an “enter decrypt password” dialog when someone tries to access the grown-up movies.
Michael
Since I got some upstream questions I’ll give an update there. The projects this would touch are ecryptfs-utils and potentially seahorse. ecryptfs-utils has stated an interest in this so that means the nautilus extension, python API, and setup/configuration UI are likely to land upstream there.

As far as seahorse I haven’t heard anything (I think I contacted them but am not sure), but I’ll hopefully meet someone involved in that project at UDS and discuss it with them. Perhaps I’ll end up putting an “Encrypted Directories” tab in that utility which allows the installation of ecryptfs-utils and from there allows you to launch the ecryptfs UI. I don’t think embedding a whole UI devoted entirely to ecryptfs makes that much sense in seahorse.
gourgi
i really like this approach!
nautilus intergration looks very cool!
As for the menu approach, i strongly believe that you should talk/work in collaboration with upstream (seahorse?) before implementing yet another system-> administration menu item. Encryption and keyrings is the right way to go IMHO…

Looking forward to have this in karmic!
Gourgi
Anonymous
What about implementing this nice feature upstream?
Wirelizard
I love the fact that the open-sourced UbuntuOne client has already help produce cool improvements like this.

All the closed-source UbuntuOne server software has produced in the same timeframe is controversy in the community!

Go open source!
Nathan Nutter
A UI is definitely necessary if desktop users are to take advantage of it. On Mac OS X there is FileVault and it is super easy to setup. However, I don’t currently use it because of concerns of stability/corruption. Of course, I have no evidence of this it’s just something that got baked into my head. I think it’s important to address the “safety” of using a new feature and I hope Canonical/Ubuntu will have some information about using encrypted home folders and/or encrypted directories.

Things like:
* How do I know some “minor” corruption won’t end up locking me out of all my data?
Michael
Thanks for all the encouraging comments everyone!

meastp, for 2-factor encryption with a USB stick I'll point you to Dustin's post on how to do exactly what you say! http://blog.dustinkirkland.com/2009/03/ubuntu-encrypted-home-with-2-factor.html

wiflye81, the theme is shiki-colors (wise) available on gnome-look and potentially will be added to the community themes package in Karmic.

Jim, I am not doing any graphical work for other desktop environments but I have created a python API for ecryptfs and a generic controller for my UI which other DEs can leverage to implement their equivalent quite easily. I'll probably be talking about this at UDS.

Philipp, filename encryption is supported since Jaunty and is the default, though you can specify to not encrypt filenames if you like.

Andrew, System -> Preferences does make sense in some ways, and ideally it would be a tab in the Encryption and Keyrings entry there, although that application is done in C++ and Glade, neither of which I am familiar with so I am not sure it will end up there. I suggested Administration simply because it is a somewhat advanced feature, and will lead to data loss if you forget your password. Though ideally this risk can be mitigated with the proper prompts to record your encryption passphrase.
Andrew
Would System > Preferences not be better, as it is on a per user basis, rather than a system wide thing.
Dread Knight
I will use encryption for sure with a GUI. :) Keep up the good work!
Philipp Kern
Are filenames encrypted nowadays?
Philipp Kern
This comment has been removed by the author.
Alan
An excellent development, but one which I might not end up using.

Reason being - I need to share an encrypted local space with Windows machines on the same network (& a MacBook possible soon …). I can (and have for some time) do this with TrueCrypt, although with less desktop integration.

So my ideal would be integrated AND Truecrypt compatible.
Jim
Hi There,

Thanks for doing this work. :) If possible, would any of this desktop integration be available for Kubuntu or Xubuntu?

Per this bug ( https://bugs.launchpad.net/ecryptfs/+bug/365796 ), the ecryptfs functionality was a bit less fleshed-out in Ubuntu’s non-GNOME distros for Jaunty, so it would be good to include Kubuntu and Xubuntu folks on this.

Thanks again!

Jim
wiflye81
Great work, love to see it in karmic.

Is this the official new theme for karmic (green looks good for koala and eucalyptus ;)) ?
Jeff
+1 for the GUI to set up encrypted directories!
MadsRH
Currently I’m not using encrypted directories, but would be more likely to if there was a slick UI to set it up.

Fantastic work :-)
Jon Dowland
I think encrypted-home-by-default for installs is a good goal to reach for. This is the first I’ve heard of ecryptfs though and I wonder if it’s the best tech for the job. I currently use libpam-mount and dm-crypt/luks.
clivew
Excellent concept
Love to see it in Karmic

Best of luck!
Anonymous
I am really looking forward to using this, especially since the new laws came in in the states saying that when you (re)enter the states they can take any paper or electronic storage device from you for “examination” without any kind of warrant or even suspicion of wrongdoing?! I mean, I have nothing illegal to hide (not even ripped music or movies) but I would prefer customs officials weren’t looking at my private documents and photos without me knowing. If they need to ask for a password, they need to give an explanation as to why they want it. :)

Anyway, I’m very grateful for your work on this, as the only thing holding me back from using an encrypted directory is a) not knowing how without a gui and b) just wanting to know it’s stable first.

Given a little time, b. will take care of itself, so I’m very happy to see work on a nice gui!

Thank you!
meastp
I would love this!

I used encrypted fs a couple of cycles ago, but it was a hassle to have to type the password on start up. Since I am using a laptop, I would like to encrypt at least some of my data, in case of theft etc. This solution will make a breeze, it seems! :)

Could you perhaps consider a feature for two-phase security (e.g passphrase-thing on a usb-pen ) on login / unlock ?
Anonymous
Great, i would love it.
Kind of EncFSVault wich replaces Apple’s FileVault with good encryption.
Hope to see this soon

Counting the number of Ubuntu users

There have been a few articles recently trying to estimate the number of Linux users, which is apparently a challenging problem. However I have to wonder why it can’t be figured out at least at the distro level by simply storing hashes of IP addresses that hit Canonical’s update site, and looking at the number of unique ones each week/month.

There are going to be people using mirrors, but this is a small percent to lose to at least get something in the right magnitude, and the most popular mirrors could probably do a similar thing and contribute their numbers anyway. The only other main drawback would be multiple Ubuntu machines under the same IP, which again seems like it would only result in a slight inaccuracy. You’d also lose a small percent to users infrequently using their computers such that they aren’t updated on a monthly basis, but yearly results would pull back in any of these people using their computers frequently enough to warrant counting.

Alternatively, as others have suggested as well, if Google would just release their numbers for browsers hitting google.com, we’d probably have a solid idea as well.

Are there already accurate numbers for Ubuntu and if not, am I missing something with my proposal?

UPDATE: Jef pointed out that Fedora is already doing this at http://fedoraproject.org/wiki/Statistics#Yum_Data, which is pretty awesome! That shows about 14 million unique repository connections, so making a VERY rough, not remotely scientific estimate, we could use distrowatch to estimate that Ubuntu has 1.68 times the number of users as Fedora, and get something around the order of 24 million users that have connected.

Comments

silver
What about universities, schools and companies using ubuntu all under the same ip-address?. This could have a pretty big influence on the numbers!
Tommi
I would either count at all the Distrowatch site. Even the sites admin says that should not be used anyway to calculate the amounts of users.

And I would like to see too that MAC address is hashed to upgrades, but Fedora is already using the smolt what generates unique ID of the system specs and sends it to Fedora. Sadly (good thing) it is not default so not all enable it. But by those figures says that Fedora has more installs than Ubuntu. Sorry about that, you ain’t special, only gaining the media attention ;-)

I dont think that market share figures means anything for us. (OK, mayby some like to extend their virtual penis).

We all - what ever distribution we use - are using same OS. It does not matter what is your opinion about system, packagemanager, brand or even freedom. All we use Linux OS to power all the other software. Without Linux, we would not be here. So kudos to Linus Torvalds to code the OS in first place and kudos for Richard Stallman to start GNU project what gave us GPL and so on free software and we got the Linux OS licended under it (GPLv2).

I see more other distributions than Ubuntu in Finland, (the homeland of the Linux OS). Same thing around on the European or Asia where I have traveled.
Ubuntu has smaller share on technical oriented users (science labs, computer stores, universities etc) but that does not matter at all. We are all using same OS, the Linux (kernel)!
scottritchie
If you want a rough estimate for the proportion of Ubuntu vs Fedora users, a better back-of-the-envelope would be the Desktop Linux Survey: http://www.desktoplinux.com/cgi-bin/survey/survey.cgi?view=archive&id=0813200712407

Now, it's 2 years out of date, but back then Fedora had 6% and Ubuntu 30%; if that's still the case today this would put Ubuntu at 5 times Fedora rather than 1.68 times Fedora.

Another metric might be Google Trends: http://www.google.com/trends?q=fedora%2C+ubuntu – suggests that the Ubuntu:Fedora ratio of searches has been increasing; so either the ratio of usage has been changing or Ubuntu users are becoming even more likely to search for Ubuntu for some reason.

Now, Google Trends and Desktop Linux surveys can be inaccurate for a variety of reasons. I'll note, however, that the google trends data is consistent with the desktop Linux survey data; both imply roughly 5x - and both are doubtlessly better proxies for usage share than distro watch.
Jef Spaleta
Micheal:
At no point did I say that all “fun” things can’t be “meaningful.”

What I said was very specific. The distrowatch metric is not meaningful. The google trends metric is not meaningful. I making absolutely no claim about the meaningfulness of any other “fun” activity. I will say that making global maps of client connections to MirrorManager using GeoIP is both “fun for me” and “meaningiful” as it gives Fedora an easy to understand snapshot of how globally used Fedora is.

You’ve extrapolated what i said and attempted to apply beyond the bounds of the original context. Is that “fun” for you as well..making gross generalization about what other people say? That’s neither friendly nor healthy. You want to keep this friendly..you want to keep this constructive? Then take more care and rein in your tendency to generalize.

I find it really amazing that you can so easily discount accurate ShipIt numbers as a useful rough metric and yet… you reached for distrowatch as a scaling metric. Stop putting the cart before the horse. Make accuracy the primary importance.. then worry about interpretation. Don’t waste your time trying to interpret the meaning of numbers that aren’t even accurate measure.

If LoCos have hundreds of cds collecting dust every release…that’s also something you could get accurate stats on…you just have to survey LoCos and ask them. if they are requesting CDs and not giving them out..that is a drain on Canonical resources. It benefits everyone by making sure that’s not happening too much.

-jef
Michael
Anonymous, hashing the MAC addresses instead is a pretty good idea! I just suggested IPs because that is something the mirrors already have access to so it wouldn’t involve any extra data sending or controversy. It is a great idea though, it would fix the cases of multiple users behind one IP, and also one user one multiple IPs.

soren, it sounds like you are right, sorry :) The more I look into this, the more I see that mirrors are more common than I thought. Thanks for enlightening me and sharing your knowledge! Though, I still think it would be feasible if enough mirrors participated. Combined with hashed MAC addresses, it has a decent accuracy potential.

Jef, ShipIt is certainly countable but I am not convinced how useful it would be. Surely some people order CDs and never use them, and someone in a shop might order one and install it a hundred times. And each LoCo could have hundreds on hand that never get used. Also, I’m not sure I could agree that something which is fun can have no meaning; fun IS meaning!

Lots of knowledgeable people have shared great stuff here, so that’s awesome! Let’s just try to keep it friendly and healthy :)
Jef Spaleta
Are you suggesting that Shipit is uncountable? It most certainly is countable. If anything its the most accurate statistic you have available to you.

Canonical could tell us tomorrow exactly the number of ShipIt disks they have paid for AND the number of disks purchased directly for the Canonical shop. Have they ever done that? Have they ever put any hard numbers out with regard to how active ShipIt is? I haven’t found them. If they haven’t that’s a pretty remarkable lack of transparency.

How about you press your leadership to publish the no-guesswork numbers associated with the amount of media sent via ShipIt for Intrepid on a monthly basis since the release of Intrepid. What is it maybe 1% of the total number of Ubuntu users Canonical employees have publicly claimed exist?

You want to haggle over a statistic that below the noise floor of any overall estimate fine..go right ahead…noise seems to be pretty important for Ubuntu supporters…much more than accuracy.
soren
Michael, I know what “order of magnitude” means, and I’d be deeply surprised if people using archive.ubunut.com as their mirror wasn’t the minority. We have an extensive mirror network, and we configure Ubuntu systems be default to use country specific mirrors. Hence, metrics based on web requests to the primary archive mirror at archive.ubuntu.com would be completely useless to even come close to the order of magnitude of users. The only thing you can use it for is to get a rough idea about trends, but even then, it’s a stretch.

Jef, you’re missing the fact that Ubuntu is freely available from ShipIt, no matter where in the world you are, and no matter what sort of network connectivity you have (as long as you can actually get to ShipIt, of course). Hence, users who can only get Linux by these means will never be counted by a service such as MirrorManager. Since a similar service to ShipIt does not (to my knowledge) exist for any other Linux distro, number from MirrorManager-like services will be biased in favour of non-Ubuntu distros. Let’s face it: There’s no way to get hard numbers for number of Linux installs.

I’m sure there’s also organisations that for whatever reason do not want to publish their use of Linux at all, even to an (alleged) anonymous service like MirrorManager. I don’t say “alleged” because I don’t believe it to be anonymous, but because there’s no way for me to know whether that’s the case or not, and for some organisations, that’s simply not good enough.
Anonymous
Couldn’t apt send the MAC-address (or some hash of it) to some kind of MirrorManager-like service? You would then get unique machines and I don’t think people would feel it would violate their privacy (I don’t feel my MAC-address is something private)…
Jef Spaleta
I’m offended.. because those are exactly the sort of numbers which the laypress picks up on. By doing the calculation and publishing a calculation you know has no merit..you are not helping make the case for solid methodology for counting that can be reused by all distros..so we can get good solid numbers out to the press. Because god knows the press don’t care about accuracy.

Do you want solid numbers or not for total linux usage? If you do, then don’t publish goofball numbers yourself.

distrowatch and google trends…while “fun” to look at..have no meaning..have no value..in any well understood sense. You might as well just generate random numbers between 1 and 10 million for all distros and call them a rough estimate with a +/- 9 million errorbar on all the numbers.

MirrorManager and the statistics it generates is a methodological approach that everyone can use..we could get solid consistent numbers across pretty much all linux distributions if they adopted the MirrorManager approach to handing out mirror information to clients dynamically. There’s real value for everyone in this tech. Users, network admins, and distributors. We don’t have to rely on CEO’s making up deployment numbers in press interviews.

-jef
Michael
Jef, I completely agree! That’s why I said it was “a VERY rough, not remotely scientific estimate”. It was just meant as a fun number. I am not at all serious about it; I thought it would be obvious from my disclaimer. Obviously the ratio obtained on distrowatch is flawed in numerous ways. I’m sorry if I have offended you by multiplying some numbers together :) If you really don’t think my disclaimer was sufficient, feel free to let me know why it isn’t VERY rough or not remotely scientific, and I can adjust it.

pfrields, thanks for leaving a comment! It is good to hear that you guys are somewhat serious about metrics like this, and are certainly quite ahead of Ubuntu (as far as I can tell) in terms of collecting the data and being open about it. Like Jef said, it is certainly an aspect that is missing from Canonical/Ubuntu which is unfortunate considering their other marketing efforts.

Anyway I definitely appreciate some Fedora folks chiming in here and didn’t mean to offend anyone with my wild extrapolations.
Jef Spaleta
Your distrowatch metric is absolute crap. I refuse to let that stand without comment.

Xandros is ranked pretty low.. and yet it has a significant number of pre-installs via being the linux Asus uses on its EEE netbooks…for like what is it now 2 whole years. In fact from netbook sales estimates Xandros is crushing Ubuntu netbook pre-installs.

In no way whatsoever can you reasonably argue that the distrowatch metric correctly places Xandros compared to the 30 or so other distros in front of it. No way.

The distrowatch scaling metric does not stand up to scrutiny.

To understand how to use the distrowatch metric you have to understand why people are going to distrowatch. You also have to understand that distrowatch’s own ranking system has a nonlinear affect on the ranking. Higher ranked distros are going to get more interests from new distrowatch visitors..because they are highly ranked. It’s a feedback loop in the methodology. And it makes for an absolutely crap metric of anything at all.

If you are serious about this you need to find a metric that actually measures what you are interested in.

What you need to do is demand Shuttleworth or any other Canonical employee who has so far been quoting userbase numbers in the press for the last 3 years..that they actually describe how they get those numbers.

http://www.theregister.co.uk/2008/10/27/shuttleworth_ubuntu_commitment/
“Precise Ubuntu installed base numbers are impossible to obtain, but Shuttleworth said the most recent estimate is about 8 million users for the Linux variant. Ubuntu does not have any call-home features to help Canonical count installations. That’s because Shuttleworth does not want to violate users’ privacy or put up any barriers to adoption for the software. “We actually have no idea,” Shuttleworth admitted.”

Numbers have contexts… methodology has meaning. You can’t just make up numbers and scaling factors just because they seem to fit the argument you are making. You have to test them for sanity. The distrowatch scaling factor is not a sane metric.
pfrields
I do look at total uniqueness once in a while. As with all the measurement scenarios, there are some flaws – the office I used to work in had many users on different versions, but they would be counted as one IP address since they were behind a firewall. And if the object is to capture number of *machine* installations, then you’re also missing users who have more than one version on separate machines at home (as I do).

Nevertheless, the count I just found from the already collected IP lists was over 12.5 million totally unique IP addresses, out of around 14 million current IP addresses found through simple summing. Obviously this count doesn’t include Fedora derivatives such as CentOS, Scientific Linux, Red Hat Enterprise Linux, and so on.
Michael
Anonymous, I am familiar with popcon but that is a purely opt-in system so as such is only useful for relative comparisons between packages, not for getting a good count.

PriceChild, using the security repo seems like a good idea, although it appears that for me I am hitting my mirror for security too. Some users/admins probably disable everything but that anyway.

Jef, that is awesome, thanks for the resource! Do you know if the total across versions is unique across them? That is, if I have Fedora 9 installed and then upgrade to Fedora 10, do I get counted twice? That seems like a common place where people are going to get double counted over the course of a year.

pochu, hopefully getting the main and US repository picks up a large percentage, and making it easy for the other mirrors to participate would make it fairly accurate.
pochu
IIRC the installer sets apt to use mirrors by default, so people using the canonical repo are likely the minority
Jef Spaleta
What you mean you don’t trust Canonical’s statement has been using every year since 2006 that Ubuntu has 8 million users? I don’t trust it either.

Fedora already has a way to estimate users via IP. We have the dynamic MirrorManager service..and it has logs…

http://fedoraproject.org/wiki/Statistics#Yum_Data

People have put a lot of thought in to what is actually achievable and what is not with regard to Fedora metrics:
http://fedoraproject.org/wiki/Infrastructure/Metrics

There’s absolutely no reason Canonical couldn’t take the MirrorManager codebase and adapt it for Ubuntu’s needs. Unlike Canonical, which spends a lot of time building closed web services codebases they are reluctant to share. All of Fedora infrastructure is done in the open..including the MirrorManager service.


MirrorManager is important enough to talk more about. Every Fedora client by default contacts the MirrorManager service asking for which mirrors to use. The MirrorManager service even lets admins on large private networks redirect fedora clients in their network block to a local private mirror..without client reconfiguration. We still count those clients because they contact MirrorManager instead of having to be manually reconfigured to point to the local mirror. Our MirrorManager service is a benefit to both the user and the local network admin who is trying to conserve bandwidth….and its enabled by default.


-jef
PriceChild
perhaps those hitting the security repo (central, not mirrored and used by default unless sa chooses not to) could be used… infact I’m pretty sure they’ve looked at that before.
Anonymous
You know that http://popcon.ubuntu.com/ exists?
Michael
soren, it doesn’t quite sound like you understand orders of magnitude if I understand your comment correctly. Unless people using the default mirror are in the MINORITY (I highly, highly doubt this is the case), it would easily be on the same order. And second, as I mentioned in the post, the most popular mirrors would likely contribute their counts to Canonical as well, making it irrelevant.

Stefano, I agree, but again this number would be better than throwing around download numbers and random guesses. And conversely to your point, as Drew mentions, there are going to be users on netbooks/laptops updating from many different IPs which will partially offset that issue.
Stefano
Yes,
counting unique IPs is the first whing that can be done, but clearly this is not accurate due to NAT.

Behind a single IP you might have N users, and you don’t know how many they are.
soren
I think you’re greatly underestimating the percentage of users using mirrors. I don’t think we’d be anywhere near the order of magnitude of users that way.
Drew Stephens
This sounds like good idea to me - each distribution’s package repositories (well, except for Red Hat/Fedora with it’s decidedly lack-luster and ill-used repos) has a good handle on the number of actively used machines. There will be some that won’t get counted, because they update from someone’s local mirror, and a few that will get counted multiple times when the check form different IPs, but it should be a close-enough number.