ZFS for linux

"Today the ZFS on Linux project reached an important milestone with the official 0.6.1 release! Over two years of use by real users has convinced us ZoL is ready for wide scale deployment on everything from desktops to super computers.

In addition to the usual bug fixes the 0.6.1 release introduces a new property called 'snapdev'. The 'snapdev' property was introduced to control the visibility of zvol snapshot devices and may be set to either 'visible' or 'hidden'. When set to 'hidden', which is the default, zvol snapshot devices will not be created under /dev/. To gain access to these devices the property must be set to 'visible' This behavior is analogous to the existing 'snapdir' property.

Other significant changes include:
  • Added Linux 3.9 compatibility
  • Added snapdev property to control visibility of zvol snapshots.
  • Disabled old on-disk format warning for zpool status -x.
  • Enabled zfs_arc_memory_throttle_disable by default.
  • Improved slab object reclaim behavior.
  • Fixed disk cache flushing for 2.6.37 and newer kernels.
  • Fixed hot spare functionality.
  • Git <id>-<hash> included in release for working builds.
  • Updated dkms and kmod compliant packaging.
  • Added man pages for splat, fsck.zfs, mount.zfs, zhack, zinject, zpios, ztest, and zpool-features."

https://groups.google.com/a/zfsonlinux.org/forum/?fromgroups=#!topic/zfs-announce/ZXADhyOwFfA

more ...

dtrace for linux

"Introduction This is a port of the Sun DTrace user and kernel code to Linux. No linux kernel code is touched in this build, but what is produced is a dynamically loadable kernel module. This avoids licensing issues and allows people to load and update dtrace as they desire. The goal of this project is to make available DTrace for the Linux platforms. By making it available for everyone, they can use it to optimise their systems and tools, and in return, I get to benefit from their work..."

https://github.com/dtrace4linux/linux

http://www.slideshare.net/brendangregg/linux-performance-analysis-and-tools

more ...

Know the wheel binary package

"Abstract

This PEP describes a built-package format for Python called "wheel".

A wheel is a ZIP-format archive with a specially formatted file name and the .whl extension. It contains a single distribution nearly as it would be installed according to PEP 376 with a particular installation scheme. Although a specialized installer is recommended, a wheel file may be installed by simply unpacking into site-packages with the standard 'unzip' tool while preserving enough information to spread its contents out onto their final paths at any later time.

PEP Acceptance

This PEP was accepted, and the defined wheel version updated to 1.0, by Nick Coghlan on 16th February, 2013 [1]

Rationale

Python needs a package format that is easier to install than sdist. Python's sdist packages are defined by and require the distutils and setuptools build systems, running arbitrary code to build-and-install, and re-compile, code just so it can be installed into a new virtualenv. This system of conflating build-install is slow, hard to maintain, and hinders innovation in both build systems and installers.

Wheel attempts to remedy these problems by providing a simpler interface between the build system and the installer. The wheel binary package format frees installers from having to know about the build system, saves time by amortizing compile time over many installations, and removes the need to install a build system in the target environment..."

http://www.python.org/dev/peps/pep-0427/

more ...

Debian Kit for Android

"This kit installs a full version of Debian or Ubuntu on your Android phone or tablet. Key features are:

  • Debian and Android side-by-side - your Android device works as usual but you can unleash the power of Debian / Ubuntu on demand.
  • Easy installation, provided that you have a fast Internet connection and you are familiar with Linux command lines to some extent.
  • You can use a compiler, install a GUI (e.g. LXDE), and run any software that is offered by Debian / Ubuntu including command line driven daemons as well as GUI software.
  • The kit is accompanied by a small software repository that contains e.g. a daemon that updates Debian's /etc/resolv.conf file from the Android net.dns property.
  • The kit runs on different hardware, i.e. armel and i386 architectures are supported. You can use a loop disk file (up to 2 Gb) as well as an extra partition on an external SD card (up to 32 Gb).
  • The Debian system does not run in a chroot jail, it's installed side-by-side to Android instead. With this, all Android files / mounts are also accessible from Debian.
  • Easy un-installation: the pre-installed Android software on your device is not altered seriously and you can get rid of this stuff if you are done. No flashing, no system extensions, no hassles..."

http://sven-ola.dyndns.org/repo/debian-kit-en.html

more ...

C++ containers that save memory and time

"C++ standard libraries commonly implement the map and set data structures using Red-Black trees, which store one element per node, requiring three pointers plus one bit of information per element to keep the tree balanced. By comparison, B-tree containers store a number of elements per node, thereby reducing pointer overhead and saving a significant amount of memory. For small data types, B-tree containers typically reduce memory use by 50 to 80% compared with Red-Black tree containers..."

http://google-opensource.blogspot.com.br/2013/01/c-containers-that-save-memory-and-time.html

more ...

Simple and Fast jobs with python-rq and rq-dashboard

"RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily..."

http://python-rq.org "rq-dashboard is a general purpose, lightweight, Flask-based web front-end to monitor your RQ queues, jobs, and workers in realtime..."

https://github.com/nvie/rq-dashboard

more ...

Cfengine3 tips - #3

With Cfengine3 you are able to define "classes" dinamically and trigger what you want to do, the following promise use this idea to install package based in information retrieved from your machine.

mkdir /etc/default/cf-tags/context
touch /etc/default/cf-tags/context/default

###

body common control {

    version => "install_with_custom_classes";

    bundlesequence => {
        "install_packages"
    };

    inputs => {
        "cfengine_stdlib.cf",
        "library.cf"
    };

}

bundle common context {

    vars:
        "context" slist =>
            lsdir("/etc/default/cf-tags/context", "", "false");

    classes:
        "context_relevance" expression => isvariable("context");

        context_relevance::
            "$(context)" expression =>
                isplain("/etc/default/cf-tags/context/$(context)");

}

bundle agent install_packages {

    methods:
        apache::
            "any" usebundle => manage_package("add", "apache2");

        bash_completion::
            "any" usebundle => manage_package("add", "bash-completion");

        postfix::
            "any" usebundle => manage_package("add", "postfix");
}

How it works?

"bundle common" is a useful place to define global variables and classes :D

bundle common context {

    vars:
        "context" slist => lsdir("/etc/default/cf-tags/context", "", "false");

Inside this example lsdir will list every file inside /etc/default/cf-tags/context and store it inside slist named context

classes:
    "context_relevance" expression => isvariable("context");

Here cfengine will verify if context is a variable and define the class context_relevance

        context_relevance::
            "$(context)" expression =>
                isplain("/etc/default/cf-tags/context/$(context)");

}

if you have context_relevance defined it will go through each value and verify if is a plain file on fs, and will declare a class with the file name if it exists 


Doing "touch /etc/default/cf-tags/context/apache" will define the apache classe to cfengine and install apache2 package. Using this idea is possible to use lots of customizations and keep your promises generic.

more ...

heatmap.js - JavaScript Library for html5 canvas based heatmaps

"What is a heatmap?

You've never heard of heatmaps? No problem! In short: a heatmap is a graphical representation of datapoints with different frequency, usually red spots on a heatmap are highly frequent spots and blue ones are less frequent spots. For a more detailed explanaition check

About heatmap.js

heatmap.js is a JavaScript library that can be used to generate web heatmaps with the html5canvas element based on your data. Heatmap instances contain a store in order to colorize the heatmap based on relative data, which means if you're adding only a single datapoint to the store it will be displayed as the hottest(red) spot, then adding another point with a higher count, it will dynamically recalculate. The heatmaps are fully customizable - you're welcome to choose your own color gradient, change its opacity, datapoint radius and many more. The library is dual-licensed under the MIT and the Beerware license, feel free to use it in your projects. Contributions on github and feedback is very appreciated..."

http://www.patrick-wied.at/static/heatmapjs http://www.patrick-wied.at/blog/the-inner-life-of-heatmap-js 

more ...

Learn python playing at www.checkio.org

"There you can learn Python coding, try yourself in solving various kinds of problems and share your ideas with others. Moreover, you can consider original solutions of other users, exchange opinions and find new friends.

If you are just starting with Python – CheckIO is a great chance for you to learn the basics and get a rich practice in solving different tasks. If you’re an experienced coder, here you’ll find an exciting opportunity to perfect your skills and learn new alternative logics from others. On CheckIO you can not only resolve the existing tasks, but also provide your own ones and even get points for them. Enjoy the possibility of playing logical games, participating in exciting competitions and share your success with friends inCheckIO.org!"

http://www.checkio.org

more ...

Cfengine3 tips - #2

Creating your own library

Even write code to automate can be repetitive, so... one thing that you are able to do with cfengine is write your own bundles to be used as method inside your policy, for example you have packages with same name on redhat and debian what could you do to avoid rewrite the package promise every time?

*vim library.cf*

bundle agent manage_package (action, package) {

    vars:
        i386|i486|i586|i686::
            "arch" slist => { "noarch", "i386", "i486", "i586", "i686" };

        x86_64::
            "arch" slist => { "noarch", "x86_64" };

    classes:
        redhat::
            "$(action)" expression => isvariable(action);

        redhat.add::
            "has_package" expression =>
                returnszero("/bin/rpm -q ${package} --quiet", "noshell");

    packages:
        redhat.!add::
            "${package}"
                package_policy => "$(action)",
                package_architectures => { "${arch}" },
                package_method => yum;

        redhat.add.!has_package::
            "${package}"
                package_policy => "$(action)",
                package_architectures => { "${arch}" },
                package_method => yum;

        debian::
            "${package}"
                package_policy => "${action}",
                package_method => apt;

}

*vim install_bash_with_library.cf *

body common control {

    version => "bash_completion_debian_0.2";

    bundlesequence => {
        "install_packages"
    };

    inputs => {
        "cfengine_stdlib.cf"
     };
}

bundle agent install_packages {
    vars:
        debian|redhat::
            "packages_to_install" slist => { "bash", "bash-completion" };

     methods:
        debian|redhat::
            "any" usebundle => manage_package("add", "$(packages_to_install)");
}

And now run:

*cf-agent -KI -f $PWD/install_bash_with_library.cf*

I will keep writing new recipes every week about cfengine3 and how to use it :D https://github.com/ncode/martinez.io-cfengine-examples

more ...