Tuesday, April 15, 2008

Are the tools developed by Fedora team tested before shipping them?

That is the question: are the tools developed by Fedora team tested before they're shipped? I'm sure that they are not tested, at least not enough, or not by normal people.

I'll be more specific to avoid being unfair. I'm talking about Revisor and livecd-tools. It's incredible. With these tools I never succeeded creating a Fedora Live CD. Never.

I'm not a newbie. I'm not stupid. I'm not lazy. I search for help on internet, forums, discussion lists - and I'm good on this. I read documentation, man pages etc. I send tickets to Bugzilla, I'm subscribed in 9 Fedora discussion lists, so you can't say I do nothing but criticize only.

Revisor is the best example. It uses some configuration files placed under /etc/revisor/conf.d/ to build installation media and/or live media. But, believe it or not, those files were incompatible with Revisor! Yes, the files shipped with the offical revisor package were incompatible with Revisor itself. Edit them, replacing $releasever and $basearch with the appropriate values and you have Revisor working fine. So, the question: why this wasn't fixed before packaging Revisor? Were Revisor checked by its packager? Were Revisor tested by its developers? If yes, I wonder what kind of tests were made and, if not, I'm going to say that Fedora is being used as a beta-testing distro by some developers/packagers.

Shipping bleeding-edge software is far different from shipping software that does not work. It's frustrating when you upgrade Fedora and a lot of software is broken. Fedora users don't expect to have absolutely stable software, but I'm sure they expect to have software that do all things done by the previous releases. Bugs in new features of a software are acceptable, but how can we understand a release that makes an essential feature stop to work, obligating you to hack your system?

At the moment, I'm trying to use livecd-creator to build a Fedora 8 Live CD, after giving up Revisor. I made a kickstart file by hand, after giving up system-config-kickstart. Generally, when you do things manually you have more chances to get the expected results, but not this time. Here are the messages:

# livecd-creator -c ~dpaulo/fedora-full.ks --cache=/srv/revisor/livecd-cache/ --fslabel=F-8-Live-1
(...)
Installing: system-config-keyboard ##################### [533/533]
umount: /var/tmp/livecd-creator-dXy9yh/install_root: device is busy
umount: /var/tmp/livecd-creator-dXy9yh/install_root: device is busy
ioctl: LOOP_CLR_FD: Device or resource busy
Traceback (most recent call last):
File "/usr/bin/livecd-creator", line 1603, in
sys.exit(main())
File "/usr/bin/livecd-creator", line 1575, in main
target.install()
File "/usr/bin/livecd-creator", line 948, in install
self.installPackages()
File "/usr/bin/livecd-creator", line 602, in installPackages
%(self.build_dir, subdir)):
OSError: [Errno 2] No such file or directory: '/var/tmp/livecd-creator-dXy9yh/install_root/etc/lvm/cache'


I'm frustrated to know that I'll have to hack my system again just because there are developers releasing buggy software and packagers accepting them. Please, Fedora team, stop to ship software that does not work. Software testing is not an obligation of end-users.

  • Update 1: for those who will say that I have to create a Bugzilla ticket, I have done that after publishing this post: https://bugzilla.redhat.com/show_bug.cgi?id=442633.
  • Update 2: for those who will say that I have do send an email to fedora-livecd-list, I have done that after publishing this post too. Read the message.
  • Update 3: contrary to what I had said earlier, software developed by the Fedora project is not bad developed. I did not want to say that. The subject of this post is software testing, not development.
  • Update 4: I've changed the title of this post from "Are the tools developed by Fedora team tested?" to "Are the tools developed by Fedora team tested before shipping them?" that express much better the subject of this post, because that are a lot of tests being done with Fedora software, and that is the reason why they are so good. I'm strictly talking about tests made before packaging a software and pushing it to the Fedora main repositories.
  • Update 5: the Revisor's bug I pointed is already closed as you can imagine once I've talked in the past as in "(...)those files were incompatible with Revisor(...)".

12 comments:

Rahul said...

Fedora Live CD's and several of the additional spins are created using livecd-creator and that by itself is enough proof that there is a lot of testing going on before release.

In livecd-creator, specifying a cache required you to create a yum-cache directory manually. This has been fixed in later revisions. Refer

https://bugzilla.redhat.com/show_bug.cgi?id=430066

Hope that helps.

Davidson Paulo said...

Rahul, the fact of Fedora Live CD's and several of the additional spins has been created using livecd-creator is a proof that there is a version of livecd-tools that works, not a proof that all its releases work pretty well. :-)

Thanks for the URL, but I have the yum-cache directory, this is not the issue.

Rahul said...

Every single version of livecd-tools released has been tested by me for the spins I maintain. If you want to contribute, posting to fedora-livecd list or filing a bug report is more helpful than presumptions. Of course there are always chances you are hitting real bugs and that is the case for any software.

Davidson Paulo said...

Rahul, the email you refer has been sent before you post your comment. I'm sure you aren't making presumptions about myself, judging me as a guy who just talk and does nothing, are you? I know what I have to do to help, and I did it. But I'm sure I can express my dissatisfaction. :-)

Anonymous said...

I've been seeing a lot of this lately. Fedora/Linux applications that are not user friendly, and support/help from the community/developers that borders on abuse.

First of all i've been using Red Hat fedora since 2001, so even though i'm not the smartest guy on the block, i manage. I've paypal contributed to several gnu projects and submitted about 30 bug reports to Fedora, doing my part helping the community.

A couple of examples of "bad user experience" was yesterday when i was helping my brother install fedora 8 over the phone. He had two drives. One drive ( drive1 ) had windows on it, and the other drive ( drive2) had his data on it. After fedora was installed on drive1 he wanted to read the data off drive2. Fedora didn't automount it, so i googled around trying to find out how to mount the drive. So i really didn't find an answer that i was happy with. So i asked on #fedora. The answer i got to get fedora to mount it was: "magic". In my opinion that guy named Opsec on #fedora should NOT be an operator. He does a disservice to fedora.

Second recent bad experience was on installing Zoneminder. After installation, i could not find any reference to Zoneminder in the Gnome GUI. The answer I got from #zoneminder was one word "init. OK its a service. But to a new user, that one word answer would be fatal. Plus, if Zoneminder requires Mysql, why is it not installed as an dependency?

So, after all these years running/supporting fedora Red Hat, i'm thinking about moving to Ubuntu to see if my user experience is any better. I imagine many developers reading this would probably say good riddance, and don't let the door hit your ass on the way out.

Anonymous said...

I haven't had any problems with LiveCD creator.

Anyhow, there is no "Fedora team" as much as there is a "Fedora community", and I'm sure testing is a huge amount of what we all do.

That being said, rather than complain about why code is not tested, how about you help test, report bugs, and get involved in the project?

Davidson Paulo said...

Anonymous, as you can see on the updates of my post, I've reported the bug on Bugzilla and sent an email to fedora-livecd-list. That was not my first bug ticket created nor my first email sent to an Fedora mailing list.

Oh yes, I am a Fedora Ambassador, translator and Editor-In-Chief of a magazine that promotes Fedora in Brazil. I'm involved in Project enough in your opinion? :-)

Davidson Paulo said...

First anonymous, I just can say you three words: "Don't do that!". Every distro has your problems. If you move in Ubuntu so you will know what is a team that don't listen to the community. Fedora have its problems, but it's moving on to the right direction, while others distributions are doing the opposite.

I was forced to install Ubuntu on mu desktop due to professional reasons, and I can't wait to get my Fedora back. :-) Fedora is much, much better, both the distro itself, the team and the community, I'm sure.

Anonymous said...

There are lots of people relying on and using livecd-creator on a regular basis and saying that is there no testing is horrendously false claim and insulting to the developers involved who are very dedicated and responsive.

If you refrain from doing that and just quietly contribute, that is a much better thing to do.

pirast said...

while I can't judge if the things that you have said are true, or are not, i have to compliment you on being very objective and not dealing too much with your emotions.

i just mean, this is how democracy and OSS work: give your opinion on something while not attacking other people and i think that your blog post has improved something. why I really have to say "go and report bugs", i think that you should talk to the developers of the tools having those bugs and work out what happened the wrong way and how to prevent those things from happening in the future.

thanks

Anonymous said...

The fact is you notice things when they're broken and you don't notice things when they just work. That's all part of being in IT. I'm not saying there's no more room for testing, there is. But I am saying we do a pretty good job of it, especially with our custom tools.

Anonymous said...

I really agree, although it is really different from package to package. NetworkManager for example is great, the X server stack is not. Bugs languish for months without attention, and from what it look like they're gong to ship F9 with an X that crashes if one uses intel and compiz at the same time, and that is being presented as a feature.