#pkgr

[geolitico]

Geheimes Manöver gegen Maaßen

Geheimes Manöver gegen Maaßen

  Hans-Georg Maaßen hat womöglich gar nicht von „linksradikalen Kräften in der SPD“ gesprochen. Was ein anonymer Brief aus seiner Behörde damit zu tun hat.

olitiker stürzen eher selten durch Angriffe ihrer politischen Gegner. Meist sind es die eigenen Leute, die zum Todesstoß ausholen. So ist es auch bei den an die Politik angegliederten Behörden, wie dem Verfassungsschutz. Weil zu Beginn der…

View On WordPress

[blog-bluesky-fan]

Bundestag muss geheime UFO-Akten freigeben

Deshalb durfte kein #AfD - Politiker in das Parlamentarische  Kontroll Gremium #PKGr ;-)

[tscheidel]

Pecha Kucha gets underway at 7:30 at San Chez. 20 slides x 20 seconds each. #pkgr (at San Chez Bistro)

[tscheidel]

Clayton Boothe explains "How to do anything" at PechaKucha. #pkgr (at San Chez Bistro)

[brinasalsa]

Lekaslah rebah, semoga pejaman matamu mengembalikan semangatmu yang menguap sehari tadi.😌

Brina

[pkgr]

Why I made Packager.io - digressions on software packaging

Last week (March 31st 2014), I launched the public beta of Packager, a service that generates debian packages for your apps using a Heroku-like workflow. It went well, and so it seems people are still interested in 'old-school' packaging.

There were good comments on the HN thread (including by docker co-founder @shykes), so I wanted to take a moment to explicit things about Packager, why it is an interesting way to distribute apps, and where it is headed. Please read on.

Where does Packager fit into the existing ways to install apps?

When it comes to the workflow for installing your apps, you're likely to fall into one of these camps:

Install everything from source (or use RVM, ruby-build, etc.) - painful, slow, and you'd better keep track of security fixes for all the stuff you're building from source. But sometimes this is the easiest way to go.

Package all your dependencies as separate debian packages (see fpm, gem2deb, etc.) - somewhat organizes things, but this is a painful process as well, and you probably need to setup and maintain an APT repository to host those packages.

Vendor everything (Omnibus) - Well, now it looks like you've got a cloned OS to maintain yourself, but you're "in control".

F*** it, I'm going to use Heroku instead. An entirely different discussion (vendor lock-in, price, privacy, etc.).

Packager fits between the 'package-all-my-dependencies' and the 'vendor-everything' approaches. It relies on the distro's packages for what could be deemed as "stable dependencies" (libc, libxml, libmysql, etc.), while the things that are most specific to your application are vendored.

For a Rails application for instance, the Ruby runtime (the specific version you asked for) and all of your gems (at the exact versions specified in your Gemfile.lock) will be embedded into the package. This is what Heroku is doing, and at Packager we're actually re-using their buildpacks (sometimes with slight variations) to generate the packages.

I think this strikes a good balance between OS packaging requirements (stable, reliable, but slow to include new stuff), and the requirements of Application packaging (fast delivery, with the best runtime available, and the latest and greatest libraries). To put it a bit differently: You don't necessarily need the latest libc available, but you certainly want to use the latest version of Rails.

Compared to Omnibus packages or installing everything from sources, this limits the security maintenance mainly to the runtime, which is definitely more manageable.

What about docker?

docker is fantastic, and it actually powers the build system of Packager. That being said:

It does not solve the installation issue. You still have to make a decision as to how you're going to install things in your docker image. Actually, docker and Packager are not mutually exclusive, as you could easily install your app in your docker container by using a package generated by Packager.

It requires additional steps to setup, and additional operational knowledge to maintain it.

Not deployed (yet) on all of your end-user machines. apt is still the easiest way to install stuff on debian-based machines.

docker is getting a lot of traction (deservedly so), but I'm convinced there is still a place for 'classic' packaging in a lot of cases.

Next steps for Packager

Reduce scope

Packager launched with many default buildpacks, most of which were not thoroughly tested to make sure they worked on all distributions. Since it is a lot of effort, for the time being I'm going to focus on the runtimes I know the best: Ruby and NodeJS. You'll still be able to specify another buildpack (see the doc), but it is most likely to fail. Additional buildpacks will be enabled once I'm sure they work reliably.

Another aspect is the target matrix, which up until now included the old stables of Ubuntu (10.04 - lucid) and Debian (6 - squeeze). Support for these distributions is dropped as of today, as I want to make sure the generated packages are working great on the current stables.

Add more apps to the showcase

I'm working on packaging well-known OpenSource Ruby and NodeJS apps, to be displayed in the showcase. Gitlab is well underway, but please contact me if you wish to get your app packaged (only Ruby and NodeJS apps for now).

Enable private projects

This is already supported by the service, for Github projects, albeit not enabled yet. However, I get the sensation that many private projects in need of Packager may not be hosted on Github. I would love to know more about your needs, so if you're interested to use Packager for packaging your private project(s), please take two minutes to answer this survey (4 questions). Thanks!

If you have further questions, please do not hesitate to say hi at [email protected], or @crohr on twitter. You should also follow @hellopkgr to get the latest news.

Update: happy to take any questions on the HN post at https://news.ycombinator.com/item?id=7546327.

[gemnasium]

Pkgr, make a package out of a Rails app in 5 minutes

What is pkgr?

Pkgr is a high-level tool that turns Rails applications into native Debian packages. Unlike fpm and omnibus, which are generic tools mostly used for packaging system software, pkgr is specialized in Rails applications.

In order to work out of the box, pkgr makes some opinionated choices for you: the applications must be git-managed and thin must be the application server. In practice, these restrictions work well enough and pkgr has even been turned into an online service, pkgr.io (currently in beta).

A sample rails application

We will start by preparing a new Rails 3.2 application (Rails 4 is still too recent), using thin as the application server.

$ mkdir PkgrTest && cd PkgrTest $ cat <<EOF > Gemfile source 'https://rubygems.org' gem 'rails', '~> 3.2.13' EOF $ bundle install $ bundle exec rails new . -d sqlite3 --skip-bundle # say yes when asked to overwrite Gemfile $ sed -i -e "s/unicorn/thin/g" -e "s/# gem 'thin'/gem 'thin'/g" Gemfile # pkgr only supports thin as app server for the time being $ sed -i -e "s/# gem 'therubyracer'/gem 'therubyracer'/g" Gemfile # we will need support for precompiled assets $ bundle $ git init . $ git add . $ git commit -m "Initial commit"

Customizing the application

Now that we have a skeleton application, let's make it a little more useful. We would like to manage our collection of videos, so we add a simple "Video" model using the Rails generator. We will also set the application routes to open the videos controller.

$ bundle exec rails generate scaffold Video title:string rating:integer notes:text $ echo >> db/seeds.rb "Video.create(title: 'Toy Story', rating: 5, notes: 'Famous for naming its characters after the Debian releases')" $ bundle exec rake db:migrate db:seed $ sed -i -e "s/# root :to.*$/root :to => 'videos#index'/" config/routes.rb $ git rm public/index.html # to use "root" action in routes.rb instead of rails placeholder $ bundle exec rails server

After performing all these steps, let's open http://localhost:3000 and check that our application is working as intended. We are still in development mode.

All good?

$ git add . $ git commit -m "added video resource"

Now let's see how easily we can package our application using pkgr.

Packaging the application

First, let's enable pkgr in our Gemfile. We will restrict it to the development group since we are not packaging anything while running in production.

$ echo "gem 'pkgr', :group => :development" >> Gemfile $ bundle $ git add Gemfile* $ git commit -m "now with pkgr"

Now we can build the application package.

$ bundle exec pkgr --uri . --bump 0.1.0 --host [email protected]

Did you notice the explicit build.host option? To build the native package, pkgr installs multiple dependencies and runs different commands, some of them as root. This is why using a disposable virtual machine for building is recommended.

We could also use localhost as the build host, as long as we are already running Debian or Ubuntu and have password-less sudo (pkgr could improve its sudo prompt detection to catch up with capistrano). This is why we explicitly specify --host [email protected], where build.host points to a disposable Debian installation.

Good to know: pkgr requires to manage the application's source code with git. The reason is that pkgr uses its own branch to inject its own configuration and support files over the application's codebase. This is quite a good idea, as this build branch can safely be removed once the packaging is done without contaminating the original code.

For experts: check the step by step instructions. It may be useful to specify extra dependencies, add custom scripts etc.

After running the build for a while, we find out that the package won't build, because pkgr used the Rails directory name ('PkgrTest') as the default package name and Debian complains about names that contains with capitalized letters. We could learn how to specify a custom package name but we are pragmatic and chose to rename the Rails directory instead.

$ cd .. $ mv PkgrTest pkgr-test $ cd pkgr-test

After rebuilding, we finally obtain a package in pkg/pkgr-test_0.1.0-1_amd64.deb.

Let's inspect it!

$ dpkg --info pkg/pkgr-test_0.1.0-1_amd64.deb new debian package, version 2.0. size 15328942 bytes: control archive= 143100 bytes. 146 bytes, 6 lines conffiles 342 bytes, 8 lines control 671952 bytes, 5057 lines md5sums 1834 bytes, 67 lines * postinst #!/bin/sh 164 bytes, 7 lines * postrm #!/bin/sh 1304 bytes, 62 lines * prerm #!/bin/sh Package: pkgr-test Version: 0.1.0-1 Architecture: amd64 Maintainer: Julien Pervillé <[email protected]> Installed-Size: 58599 Depends: libc6 (>= 2.15), libgcc1 (>= 1:4.1.1), libruby1.9.1 (>= 1.9.3~rc1), libsqlite3-0 (>= 3.6.11), libstdc++6 (>= 4.1.1), ruby1.9.1-full, git-core, libxml2, libxslt1.1 Section: unknown Priority: extra $ dpkg --contents pkg/pkgr-test_0.1.0-1_amd64.deb # ... lots of files listed drwxr-xr-x root/root 0 2013-07-20 00:15 ./var/log/ drwxr-xr-x root/root 0 2013-07-20 00:15 ./var/log/pkgr-test/ lrwxrwxrwx root/root 0 2013-07-20 00:15 ./opt/local/pkgr-test/db -> /var/db/pkgr-test lrwxrwxrwx root/root 0 2013-07-20 00:15 ./opt/local/pkgr-test/log -> /var/log/pkgr-test lrwxrwxrwx root/root 0 2013-07-20 00:15 ./opt/local/pkgr-test/config/pkgr.yml -> /etc/pkgr-test/pkgr.yml lrwxrwxrwx root/root 0 2013-07-20 00:15 ./opt/local/pkgr-test/config/database.yml -> /etc/pkgr-test/database.yml

Let's see if the new package installs on a clean machine.

$ scp pkg/pkgr-test_0.1.0-1_amd64.deb [email protected]: $ ssh [email protected] 'sudo dpkg -i pkgr-test_0.1.0-1_amd64.deb || sudo apt-get -f -y install'

Open http://production.host:8000 (pkgr configures the application server to listen on port 8000 by default). We are welcomed by a "500" error page. A quick look at /var/log/pkgr-test on the production target shows us that we need to setup the database.

$ ssh [email protected] 'sudo pkgr-test rake db:migrate' $ ssh [email protected] 'sudo pkgr-test server restart'

Since we have experience with Rails, we use the opportunity to precompile the assets before getting another "500" error.

# sudo -u pkgr-test sh -c 'cd /opt/local/pkgr-test && bundle exec rake assets:precompile'

Please note: there is a pkgr branch to support asset precompilation but not merged yet.

Integrating into a virtualhost

pkgr packages the Rails application, creates the application user, setup directories, log rotation and init scripts, but it does not configure the frontend web server (apache or nginx) to forward traffic to it.

To finish the job, we will configure an nginx virtualhost for the application. I don't know yet if such a virtualhost can be easily added to pkgr but it would be a nice feature to have, for sure.

# on the production host $ sudo apt-get install nginx $ sudo cat <<EOF > /etc/nginx/sites-available/pkgr-test server { listen *:80 default_server; #server_name pkgr-test.example.com; server_tokens off; # don't show the version number, a security best practice root /opt/local/pkgr-test/public; # individual nginx logs for this gitlab vhost access_log /var/log/pkgr-test/nginx_access.log; error_log /var/log/pkgr-test/nginx_error.log; location / { # serve static files from defined root folder;. try_files $uri \$uri/index.html \$uri.html @upstream; } location @upstream { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header X-Forwarded-Proto \$scheme; proxy_set_header Host \$http_host; proxy_set_header X-Real-IP \$remote_addr; proxy_pass http://localhost:8000; } # if the request is for a static resource, nginx should serve it directly # and add a far future expires header to it, making the browser # cache the resource and navigate faster over the website. location /assets { expires max; add_header Cache-Control public; break; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } EOF $ sudo rm /etc/nginx/sites-enabled/default $ sudo ln -s /etc/nginx/sites-available/pkgr-test /etc/nginx/sites-enabled/pkgr-test $ sudo service nginx restart

Now we can go back to our browser and enjoy the application in production. If we update the application's code, we can run a pkgr build again, but it should go much faster.

We found a few bugs and gotchas in the process and it took more than 5 minutes to turn the Rails application into a package, but we still saved us a lot of time and had fun while at it. Thank you pkgr!

Conclusion

I liked the idea of easily building a native package from a Rails application. I also appreciate that pkgr provides default control and init scripts, logrotate rules etc. to make the Rails application play well with the operating system.

However, I believe that the choices taken by the author are a bit limiting and that more potential for customization would be useful. I'd love to have a better control script, use puma as application server or add monit rules to watch my application as I install it.

The build process could also be more robust, as I had to struggle quite a lot to build my packages (sudo prompt detection issues, network issues during downloads etc). In general, the build script only works in the happy path and is completely lost when one of the steps fails.

I'm sure that, with some ironing out, pkgr could have a lot of potential and I'm willing to contribute some pull requests.

[kreuzbergpredigt]

Members of the JLG (Justice League of Germany) 

#1 Martian Ströbele

Origin: Halle auf dem Mars

Tagline: "Gebt das Hanf frei!" 

Super-Power: Unterstützung von kriminellen Vereinigungen

Position in formation: linker Flügel