Sections

Skip to content. | Skip to navigation

You are here: Home Blog Topics python

python

Apr 25, 2008

pyJigdo - v0.3.0 Released

Filed Under:

pyJigdo version 0.3.0 has been sent off to the Fedora build system. There has been a lot of work put into this release to make it a stable starting ground for everything we want to achieve with pyJigdo.

pyJigdo Logo

No more jigdo-lite...

I'm delighted to inform the community that we now have a good alternative to jigdo-lite for downloading the up-coming Fedora [jigdo] release. There has been a good amount of testing that has gone into this release. This amount of testing, coupled with the complete rewrite, has proven to be quite fruitful. Two [important] things that remain to be tested are running this release on F7 and F9; I have done all my testing with F8. This release already has many more features then jigdo-lite and, in most cases, saves time. Some of the more notable new features include the ability to auto-mount an existing ISO image (via fuseiso,) the ability to efficiently search directories for needed files, and the ability to use mirror lists. See pyjigdo --help for all of the currently available features.

Where to download?

I've just sent out the builds, so if you don't want to wait for it to hit updates-testing (and then updates) go to the pyjigdo koji page for builds. If you do end up testing it, please mark your comments via Bodhi for F7 and F8. This release should be yum installable soon enough, however.

Where do bugs go?

Please file bugs either in the redhat bugzilla, or preferably on the Fedora Hosted pyJigdo trac instance.

What does the future bring?

We have a lot planned for pyJigdo, so please keep an eye out for more releases (or send patches for features and fixes.) Also, watch for changes to our roadmap as I am going to try to get everything/anything I plan on doing into trac before I start working on it.

 

Feb 20, 2008

pyJigdo - Making Jigdo Work

Filed Under:

Jigdo has been around for a while and has proven itself useful for Debian. I found most of the resistance to using it in Fedora stemming from the fact the client is not amazing. Enter pyJigdo...

How is this Related to Fedora?

In the Fedora universe, many things have been done to open up the distribution and have made developing Fedora very interesting. One of the new concepts is Re-Spinning the distribution for specific use cases. Many companies don't have the resources to compose their own in-house distribution and share it network wide but they do have a use case that warrants a "corporate standard" desktop that is maintainable by as little as one person. Fedora users are now able to take the published packages and Re-Mix, so to speak, the package universe and create something specific to their use cases. This includes customizing runtime settings, available packages and even making their desktops (even servers) stateless. A single system administrator can easily create their own flavor of Fedora (or even a derivative) with a few simple clicks of a mouse or minor adjustments to kickstart definitions which can then be shared company wide. Not only will this increase the likelihood of more people trying Fedora, it will create a more stable and thus more productive environment. It will also lead to IT staff having more time to focus on business applications of their technology rather then running Spyware, Malware and Anti-Virus software wasting time and computational resources. There is so much more about Re-Spins and Re-Mixes but I have to stop as it is outside of the scope of this particular blog posting.

What is Jigdo?

"Jigsaw Download, or short jigdo, is a tool designed to ease the distribution of very large files over the internet, for example CD or DVD images. Its aim is to make downloading the images as easy for users as a click on a direct download link in a browser, while avoiding all the problems that server administrators have with hosting such large files."

http://atterer.net/jigdo/

How does this help?

When having to download a large file, such as a full ISO image, from a single mirror users can run into slow link speeds, corrupted downloads and wasted time. Even more so, if needing to share a given image or image set to many locations the amount of time to transfer from a single location is greatly increased as more locations are requesting data. Jigdo provides a mechanism to create a "definition" of a given image. This definition can easily be shared and is trivial in size in comparison to a full image. The jigdo definition enables consumers of the image (for lack of a better term) to put back together the image(s) easily and efficiently. Most of the efficiency comes from the ability to use multiple sources to fetch data including local data sources or existing images. In the case of a Re-Spin, a jigdo definition can be used to "patch" a past Spin resulting in a fully updated image. In the case of a Re-Mix, packages that make up the image can be fetched from many sources including an on-site install tree (normally used to do PXE or network based installations) or even a system such as cobbler . Also, all files/data that make up the image are hashed and will eliminate corrupt images which waste time and bandwidth. The inherit format of the definition also provides a healthy layer of transparency as to the contents of the resulting image. In my humble opinion, there is many more benefits but in the interest of being terse I continue.

What about BitTorrent?

There is nothing wrong with bittorrent and it provides many of the same benefits as jigdo does. One of the major complaints about bittorrent is the inability to use it on some network infrastructure. Not only does one need to run a "tracker" to keep tabs on peers but "seeds" have to run BitTorrent software. It's not an everyday thing where mirror administrators (those with serious servers and serious bandwidth) are willing to fire up a torrent client for a customized flavor of a distribution they already mirror or even a full/official release. BitTorrent has many viable use cases and I concede that there are good arguments for using BitTorrent in the use cases outlined above. However, I don't believe it is the best solution.

Where to Next?

As Jigdo is almost 100% client side, we need to make a better client. jigdo-lite (a shell script) has served it's purpose but we need to create a more extensible and maintainable client. As a result of these needs, the pyJigdo code base has been created. We need interested python developers to help with the effort of both creating a fast and efficient implementation of jigdo in python and creating an interface (both CLI and GUI) that enables users to create, host, assemble, verify, [insert your feature here] and inspect jigdo definitions and templates. Development efforts will continue but to succeed we will need developers passionate about what concepts Jigdo presents.

Okay, so How do I Help?