Skip to content

Python-3.4 sur ma Gentoo habituée à Python-2.7

A la découverte de Python-3.4.0 sur une de mes install Gentoo :) A ce jour (17/05/2014) Python-3.4 est encore masqué sous Gentoo, je vais l’installer sans modifier (volontairement) le package.keywords :

ACCEPT_KEYWORDS="~amd64" emerge -av =python-3.4*

[ebuild  NS    ] dev-lang/python-3.4.0:3.4 [2.7.5-r3:2.7, 3.3.3:3.3] USE="gdbm ipv6 ncurses readline sqlite ssl threads xml -build -examples -hardened -tk -wininst" 13,768 kB
GNU/Linux Magazine 171 (Mai 2014)

GNU/Linux Magazine 171 (Mai 2014)

J’utilisais jusque là python-2.7 uniquement, entendant ici ou là (notamment avec les développeurs Ansible) que Python >= 3.0 s’apparentait plutôt à un nouveau language qu’à une évolution de ce dernier.

C’est en lisant GNU/Linux Magazine 171 (Mai 2014) que je me retrouve tenté d’essayer la version 3.4  - entre-autres pour le module asyncio…

J’en profite par ailleurs pour parler de ce “Dr KissCool” visiblement adepte du troll et accompagné d’un humour de second et troisième degré. Je n’aime pas ce rédacteur, je ne remet pas en cause ses éventuels talents dans le domaine, mais je ne prends presque aucun plaisir à lire son blah blah trop souvent éloigné à mon goût de l’information technique recherchée dans ce genre de Mag.

Mes contenus sur ce blog sont probablement moins pertinents avec plus de fautes d’orthographes et grammaticales, mais personne ne paye pour les lires

Je n’oublie pas ensuite d’aller modifier le PYTHON_TARGETS dans make.conf que Gentoo consultera pour déterminer les version de python concernées par l’installation de nouveaux modules  :

PYTHON_TARGETS="python2_7 python3_3 python3_4"

Il faudrait ensuite théoriquement demander à Gentoo la recompilation de tous les modules python pour prendre en compte cette nouvelle branche :

 * You have just upgraded from an older version of Python.
 *
 * Please adjust PYTHON_TARGETS (if so desired), and run emerge with the --newuse or --changed-use option to rebuild packages installing python modules.

Tiens donc, jusque là, Gentoo préconisait l’utilisation de l’outil python-updater pour ça ; mais effectivement le résultat n’a pas l’air d’être pertinent :

# python-updater -- --pretend --tree --verbose
 * Starting Python Updater...
 * Main active version of Python:    2.7
 * Active version of Python 2:       2.7
 * Active version of Python 3:       3.3
 * Globally supported Python ABIs in installed repositories:
 *   gentoo:                         2.4 2.5 2.6 2.7 3.1 3.2 3.3 2.5-jython 2.7-jython 2.7-pypy-1.7 2.7-pypy-1.8 2.7-pypy-1.9 2.7-pypy-2.0
 *   sdfoverlay:                     2.4 2.5 2.6 2.7 3.1 3.2 3.3 2.5-jython 2.7-jython 2.7-pypy-1.7 2.7-pypy-1.8 2.7-pypy-1.9 2.7-pypy-2.0
 *   Adding to list: sys-libs/cracklib:0
 * emerge -Dv1 --keep-going sys-libs/cracklib:0 --pretend --tree --verbose

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild     U  ] sys-libs/cracklib-2.9.1 [2.8.19] USE="nls zlib -python -static-libs {-test%} (-build%)" PYTHON_TARGETS="python2_7%* (-python2_6)" 621 kB

Total: 1 package (1 upgrade), Size of downloads: 621 kB

par ailleurs, on constate que python-3.4 ne semble pas être listé dans les “Globally supported ABIs” ; cela à t’il donc un intérêt de recompiler les modules avec emerge ? essayons avec dev-python/django pour commencer :

emerge -av dev-python/django

[ebuild     U  ] dev-python/django-1.4.11 [1.4.8] USE="vhosts -mysql -postgres -sqlite {-test} (-doc%)" PYTHON_TARGETS="python2_7 (-python2_6%)" 7,571 kB
# equery files dev-python/django | grep 2.7 | tail -n 1
/usr/lib64/python2.7/site-packages/django/views/static.pyo
#
# equery files dev-python/django | grep 3.4 | tail -n 1
# 

à priori non, alors je m’arrête là et j’utilise python3.4 pour mes tests sans me soucier (et sans que cela ne perturbe) mes install stable via portage de python2.7 et python3.4 … Désolé si ce post est… inutile…

# python3.4
Python 3.4.0 (default, May 17 2014, 11:59:14)
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Preparing a new Rig (Gentoo + SGMiner)

bios-z77a-g45This page is used with a MSI Z77A-G45 Motherboard UEFI, a SSD hard drive SATA partionned with GPT,

1. Update the motherboard firmware (aka BIOS even if UEFI based)

2.Configure the motherboard

  1. Settings > Advanced > Power Management Setup > Restore after AC power loss : [Power On]
  2. Settings > Boot > Full screen logo display : [Disabled]
  3. Settings > Boot > Boot mode select : [legacy+uefi]
  4. Settings > Boot > Forced boot order :
    1. UEFI USB Key
    2. USB Key
    3. UEFI USB CD/DVD
    4. USB CD/DVD
    5. UEFI Hard Disk
    6. powerHard Disk

3.  Configure your modem/router :

  1. Set a static DHCP entry for the mac address of the motherboard, ie 192.168.0.100
  2. Set a port forwarding from 22100/tcp to 22100/tcp

4. Boot on a Gentoo Install Minimal USB Key :

  1. Set a temporary root password (“passwd”)
  2. Set “Port 22100″ in /etc/ssh/sshd_config
  3. Launch sshd : “/etc/init.d/sshd start”
  4. Check with “ip a” that eth0 have the good ip address (may be enp2s0, enp3s0 or other)

5. Prepare playbook execution

  1. Connect with ssh : “ssh -p 22100 root@192.168.0.100″
  2. Add your ssh public key to /root/.ssh/authorized_keys
  3. Retry the connection with the ssh key
  4. disable or change the root passwd with a huge one
  5. load the kernel module for efi : “modprobe efivars”
  6. Partition, format and mount the harddrive :
# partition
parted --script --align optimal /dev/sda mklabel gpt
parted --script --align optimal /dev/sda mklabel msdos
parted --script --align optimal /dev/sda mklabel gpt
parted --script --align optimal /dev/sda mkpart primary fat32 0% 256MiB
parted --script --align optimal /dev/sda mkpart primary ext2 256MiB 512MiB
parted --script --align optimal /dev/sda mkpart primary linux-swap 512MiB 768MiB
parted --script --align optimal /dev/sda mkpart primary ext3 768MiB 100%
parted --script --align optimal /dev/sda set 1 boot on
parted --script --align optimal /dev/sda print

# format
mkfs.vfat -F32 -s 2 -n efi-boot /dev/sda1
mkfs.ext2 /dev/sda2
mkfs.ext3 /dev/sda4
mkswap /dev/sda3

# mount
swapon /dev/sda3
mount /dev/sda4 /mnt/gentoo
mkdir /mnt/gentoo/boot
mount /dev/sda2 /mnt/gentoo/boot
mkdir /mnt/gentoo/boot/efi
mount /dev/sda1 /mnt/gentoo/boot/efi

6. Execute the playbook to install Gentoo stage3 (amd64), portage, base configuration, grub, programs, x, drivers, kernel

many shell / emerge commands in this playbook waiting for me to finish the emerge module for ansible :

- hosts: gentoo_stage3_chroot_and_mining_base

  vars:
    chroot_basedir: "/mnt/gentoo"

  vars_files:
    - ../global_vars.yml

  user: root

  tasks:

    - copy: backup=no src=${basedir}/files/gentoo/${gentoo_stage3_latest} dest=${chroot_basedir}/${gentoo_stage3_latest} owner=root group=root mode=0644
    - copy: backup=no src=${basedir}/files/gentoo/${gentoo_portage_latest} dest=${chroot_basedir}/${gentoo_portage_latest} owner=root group=root mode=0644
    - file: path=${chroot_basedir} owner=root group=root mode=0755 state=directory
    - shell: test -d ${chroot_basedir}/usr || tar xf ${chroot_basedir}/${gentoo_stage3_latest} -C ${chroot_basedir}
    - shell: test -d ${chroot_basedir}/usr/portage || tar xf ${chroot_basedir}/${gentoo_portage_latest} -C ${chroot_basedir}/usr

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /tmp && exec /etc/init.d/sshd checkconfig" &>/dev/null
    - file: path=${chroot_basedir}/root/.ssh owner=root group=root mode=0700 state=directory
    - shell: test -e ${chroot_basedir}/root/.ssh/authorized_keys || touch ${chroot_basedir}/root/.ssh/authorized_keys
    - file: path=${chroot_basedir}/root/.ssh/authorized_keys owner=root group=root mode=0600 state=file
    - lineinfile: dest=${chroot_basedir}/root/.ssh/authorized_keys regexp='w8TbqlGQqcs0WcvHGkTuI58fUTX50QPqz54YVBYK1m6zj3sLM2uJPPPr2' line='from="176.123.63.0/24,192.168.0.0/24,192.168.1.0/24" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpJ+UQL3U7jeEJmJFmtfEz69sfFiqXaX/9z2EGx1ZFDxgyUVZ5Kyn5wrVbEufExQ1D3N8PINLRg7tL0JArofiJCVcBqEWk/w8TbqlGQqcs0WcvHGkTuI58fUTX50QPqz54YVBYK1m6zj3sLM2uJPPPr2r/6FZm8S1kxfMhtO6n/8q47M4YzXp5cHwzgejcLNjez5RS5Vc+EA+QVynOgNFrCSGhPEI5P1tSXTLFmD9kqBvbMTKdTQC4ao+yu2xkyudzg2SBFcnvDzfvKkfDEqy15gjI9fCfd+Glvg4BHpvx0re7Gx4E+wGAv6ZxWHclGNPgGhj4o+hPOwlcV+E6PcVT davixx@madness.davixx.net'

    - shell: cp -L /etc/resolv.conf ${chroot_basedir}/etc/resolv.conf
    - shell: mount | grep "${chroot_basedir}/proc" &>/dev/null || mount -t proc none ${chroot_basedir}/proc
    - shell: mount | grep "${chroot_basedir}/dev" &>/dev/null || mount --rbind /dev ${chroot_basedir}/dev
    - shell: mount | grep "${chroot_basedir}/sys" &>/dev/null || mount --rbind /sys ${chroot_basedir}/sys
    - shell: test -e ${chroot_basedir}/usr/bin/python2.7 || chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /tmp && exec /usr/bin/emerge --quiet dev-lang/python:2.7" &>/dev/null
    - shell: ${chroot_basedir}/usr/bin/python -V | grep 'Python 2.7.' &>/dev/null || chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /tmp && exec eselect python set python2.7" &>/dev/null

    - shell: cp -L /etc/resolv.conf ${chroot_basedir}/etc/resolv.conf

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /tmp && exec /usr/bin/emerge --sync" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "cp /usr/share/zoneinfo/Europe/Paris /etc/localtime" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "echo 'Europe/Paris' > /etc/timezone" &>/dev/null

    # todo hour / date 012514422014 # MMJJhhmmAAAA

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec eselect profile set default/linux/amd64/13.0" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "echo '127.0.0.1 ${host}.davixx.net ${host} localhost' > /etc/hosts" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "echo 'hostname=${host}' > /etc/conf.d/hostname" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec hostname ${host}" &>/dev/null

    - template: src=${basedir}/templates/net_eth0 dest=${chroot_basedir}/etc/conf.d/net owner=root group=root mode=0644
    - shell: chroot ${chroot_basedir} /bin/bash -c "cd /etc/init.d && exec ln -snf net.lo net.eth0" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec rc-update add net.eth0 default" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec rc-update add sshd default" &>/dev/null

    - lineinfile: dest=${chroot_basedir}/etc/conf.d/keymaps regexp='^keymap=' line='keymap="fr"'

    - lineinfile: dest=${chroot_basedir}/etc/portage/make.conf regexp='^USE=' line='USE="-bindist mmx sse sse2"'
    - lineinfile: dest=${chroot_basedir}/etc/portage/make.conf regexp='^GRUB_PLATFORMS=' line='GRUB_PLATFORMS="efi-64"'
    - lineinfile: dest=${chroot_basedir}/etc/portage/make.conf regexp='^VIDEO_CARDS=' line='VIDEO_CARDS="fglrx vesa"'
    - lineinfile: dest=${chroot_basedir}/etc/portage/make.conf regexp='^INPUT_DEVICES=' line='INPUT_DEVICES="evdev"'

    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^x11-wm/fluxbox\s' line='x11-wm/fluxbox -gnome -kde truetype vim-syntax' create=yes
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^x11-libs/cairo\s' line='x11-libs/cairo X'
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^media-libs/imlib2\s' line='media-libs/imlib2 X'
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^x11-libs/libdrm\s' line='x11-libs/libdrm libkms'
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^media-libs/mesa\s' line='media-libs/mesa xa'
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^dev-libs/libxml2\s' line='dev-libs/libxml2 python'
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^x11-base/xorg-server\s' line='x11-base/xorg-server udev'
    - lineinfile: dest=${chroot_basedir}/etc/portage/package.use regexp='^x11-libs/pango\s' line='x11-libs/pango X'

    - lineinfile: dest=${chroot_basedir}/etc/portage/package.keywords regexp='^=x11-drivers/ati-drivers-' line='=x11-drivers/ati-drivers-13.9' create=yes

    - template: src=${basedir}/templates/fstab_gentoo_efi_grub2 dest=${chroot_basedir}/etc/fstab owner=root group=root mode=0644

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn --newuse dev-libs/openssl net-misc/openssh" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn =sys-kernel/gentoo-sources-3.10.25 sys-kernel/linux-firmware sys-fs/dosfstools sys-block/parted net-analyzer/tcpdump" &>/dev/null

    - copy: backup=no src=${basedir}/files/kernel/${kernel_image_latest} dest=${chroot_basedir}/boot/${kernel_image_latest} owner=root group=root mode=0644
    - copy: backup=no src=${basedir}/files/kernel/${kernel_config_latest} dest=${chroot_basedir}/boot/${kernel_config_latest} owner=root group=root mode=0644
    - copy: backup=no src=${basedir}/files/kernel/${kernel_config_latest} dest=${chroot_basedir}/usr/src/linux-${kernel_version_latest}/.config owner=root group=root mode=0644
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /usr/src/linux-${kernel_version_latest} && exec make -j3" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /usr/src/linux-${kernel_version_latest} && exec make -j3 modules modules_install" &>/dev/null
    - copy: backup=no src=${basedir}/files/drivers/${driver_r8168_tarball} dest=${chroot_basedir}/usr/portage/distfiles/${driver_r8168_tarball} owner=root group=root mode=0644
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && cd /tmp && rm -rf ./${driver_r8168_dirname} && tar xf /usr/portage/distfiles/${driver_r8168_tarball} && cd ./${driver_r8168_dirname} && make clean modules && exec make install" &>/dev/null
    - lineinfile: dest=${chroot_basedir}/etc/conf.d/modules regexp='^modules=' line='modules="${modules} r8168"'

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn app-editors/vim app-admin/metalog sys-process/fcron net-misc/openvpn dev-vcs/git dev-vcs/subversion app-admin/pwgen" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn net-misc/dhcpcd sys-boot/grub app-portage/gentoolkit dev-util/strace sys-apps/iproute2 app-misc/screen app-portage/eix" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn x11-base/xorg-server fluxbox xlockmore eterm x11-misc/slim x11-apps/xdm" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn commonbox-styles commonbox-styles-extra fluxbox-styles-fluxmod gentoo-artwork xf86-input-evdev xf86-video-fbdev app-admin/conky" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn dev-python/pip dev-python/setuptools sys-process/daemontools sys-process/supervise-scripts sys-apps/lm_sensors" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec /usr/bin/emerge -qn net-libs/nodejs net-dns/unbound www-client/links net-misc/openntpd app-arch/rpm2targz x11-libs/wxGTK app-admin/sudo x11-terms/xterm net-misc/telnet-bsd" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec eselect news read all" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec svscan-add-to-inittab" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec emerge --config sys-process/fcron" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec emerge --config mail-mta/nullmailer" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec rc-update add metalog default" &>/dev/null
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec rc-update add fcron default" &>/dev/null

    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec eix-update" &>/dev/null

    - lineinfile: dest=${chroot_basedir}/etc/ssh/sshd_config regexp='^Port\s+' line="Port ${ansible_ssh_port}"

    - lineinfile: dest=${chroot_basedir}/etc/default/grub regexp='^GRUB_TIMEOUT=' line='GRUB_TIMEOUT=3'
    - lineinfile: dest=${chroot_basedir}/etc/default/grub regexp='^GRUB_CMDLINE_LINUX=' line='GRUB_CMDLINE_LINUX="net.ifnames=0"'
    - file: path=${chroot_basedir}/boot/grub state=directory
    - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec grub2-mkconfig -o /boot/grub/grub.cfg" &>/dev/null
    # - shell: chroot ${chroot_basedir} /bin/bash -c "env-update && source /etc/profile && exec grub2-install --efi-directory=/boot/efi --target=x86_64-efi /dev/sda --recheck" &>/dev/null

    - copy: backup=no src=${basedir}/templates/openvpn_conf_MINING01 dest=${chroot_basedir}/etc/openvpn/openvpn.conf owner=root group=root mode=0600
    - copy: backup=no src=${basedir}/files/vpn/MINING01/ca.crt dest=${chroot_basedir}/etc/openvpn/ca.crt owner=root group=root mode=0600
    - copy: backup=no src=${basedir}/files/vpn/MINING01/ta.key dest=${chroot_basedir}/etc/openvpn/ta.key owner=root group=root mode=0600
    - copy: backup=no src=${basedir}/files/vpn/MINING01/${host}.crt dest=${chroot_basedir}/etc/openvpn/server.crt owner=root group=root mode=0600
    - copy: backup=no src=${basedir}/files/vpn/MINING01/${host}.key dest=${chroot_basedir}/etc/openvpn/server.key owner=root group=root mode=0600

    - file: path=${chroot_basedir}/root/services state=directory

    - file: path=${chroot_basedir}/root/services/ntpd state=directory
    - copy: backup=no src=${basedir}/files/services/ntpd/run dest=${chroot_basedir}/root/services/ntpd/run owner=root group=root mode=0700
    - shell: chroot ${chroot_basedir} /bin/bash -c "ln -snf /root/services/ntpd /service/ntpd" &>/dev/null
    - file: path=${chroot_basedir}/etc/unbound/unbound.conf state=absent

    - file: path=${chroot_basedir}/root/services/unbound state=directory
    - copy: backup=no src=${basedir}/files/services/unbound/run dest=${chroot_basedir}/root/services/unbound/run owner=root group=root mode=0700
    - shell: chroot ${chroot_basedir} /bin/bash -c "ln -snf /root/services/unbound /service/unbound" &>/dev/null

    - file: path=${chroot_basedir}/root/services/openvpn state=directory
    - copy: backup=no src=${basedir}/files/services/openvpn/run dest=${chroot_basedir}/root/services/openvpn/run owner=root group=root mode=0700
    - shell: chroot ${chroot_basedir} /bin/bash -c "ln -snf /root/services/openvpn /service/openvpn" &>/dev/null

    - lineinfile: dest=${chroot_basedir}/etc/conf.d/xdm regexp='^DISPLAYMANAGER=' line='DISPLAYMANAGER="slim"'

    - file: path=${chroot_basedir}/${gentoo_stage3_latest} state=absent
    - file: path=${chroot_basedir}/${gentoo_portage_latest} state=absent

    - shell: umount ${chroot_basedir}/proc
    - shell: umount ${chroot_basedir}/dev/pts || true
    - shell: umount ${chroot_basedir}/dev || true
    - shell: umount ${chroot_basedir}/sys || true
    - shell: umount ${chroot_basedir}/boot/efi
    - shell: umount ${chroot_basedir}/boot
    - shell: mount -oremount,ro /mnt/gentoo

7. Manual step (because of the fucking usb key who virtualize the gentoo cd install, and hide uefi hardware)

# mount (or remount) disk
mount /dev/sda4 /mnt/gentoo
mount -oremount,rw /mnt/gentoo
mount /dev/sda2 /mnt/gentoo/boot
mount /dev/sda1 /mnt/gentoo/boot/efi
mount -t proc proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev

# chroot
chroot /mnt/gentoo /bin/bash

# init env
env-update && source /etc/profile

# plug an usb key (check it with dmesg / dmesg -c)
# suppose she can be erased, and named sdc

# partition it !
parted --script --align optimal /dev/sdc mklabel msdos
parted --script --align optimal /dev/sdc mklabel gpt
parted --script --align optimal /dev/sdc mklabel msdos
parted --script --align optimal /dev/sdc mkpart primary fat32 0% 100%
parted --script --align optimal /dev/sdc set 1 boot on
parted --script --align optimal /dev/sdc print

# format it
mkfs.vfat -F32 -s 2 -n efi-boot /dev/sdc1

# mount it
mount /dev/sdc1 /media

# install grub on it !
grub2-install --target=x86_64-efi --efi-directory=/media --removable --modules=part_gpt

# umount it !
umount /media

# optionally set a static password (needed if something goes wrong at boot)
passwd

# leave the chroot
exit

# out of the chroot now

# umount disk (dev and sys will probably fail)
umount /mnt/gentoo/dev
umount /mnt/gentoo/sys
umount /mnt/gentoo/proc
umount /mnt/gentoo/boot/efi
umount /mnt/gentoo/boot
mount -oremount,ro /mnt/gentoo

8.  Check, and write grub on disk (to be able to definitly not use anymore the usb efi key

grub2-install --efi-directory=/boot/efi --target=x86_64-efi /dev/sda --recheck

also check :

  1. if VPN connection is up
  2. if unbound is ready and used in resolv.conf
  3. ntpd is up, and system is ntp synchronised

9. execute the mining playbook !

  1. move the rig in the [mining01] inventory place
  2. execute the mining01.yml playbook

 

- hosts: mining01

  vars:
    sgminer_tarball: sgminer-${sgminer_version}.tar.gz
    amdadl_tarball: ADL_SDK_${admadl_version}.zip

  vars_files:
    - ../global_vars.yml

  user: root

  tasks:

    - copy: backup=no src=${basedir}/files/distfiles/${amdadl_tarball} dest=/usr/portage/distfiles/${amdadl_tarball} owner=root group=root mode=0644
    - shell: emerge -qn x11-libs/amd-adl-sdk &>/dev/null
    - shell: bash -c "eselect opengl list | grep -- 'ati \*' || eselect opengl set ati"  &>/dev/null

    - file: path=/root/services/xinit state=directory
    - copy: backup=no src=${basedir}/files/services/xinit/run dest=/root/services/xinit/run owner=root group=root mode=0700
    - file: src=/root/services/xinit dest=/service/xinit owner=root group=root state=link

    - copy: backup=no src=${basedir}/files/distfiles/${sgminer_tarball} dest=/usr/portage/distfiles/${sgminer_tarball} owner=root group=root mode=0644
    - file: path=/root/services/sgminer state=directory
    - file: path=/root/.sgminer state=directory
    - copy: backup=no src=${basedir}/files/services/sgminer/sgminer_${host}.conf dest=/root/.sgminer/sgminer.conf owner=root group=root mode=0600
    - copy: backup=no src=${basedir}/files/services/sgminer/build.sh dest=/root/services/sgminer/build.sh owner=root group=root mode=0700
    - shell: /root/services/sgminer/build.sh ${sgminer_version} ${admadl_version} &>/dev/null
    - template: src=${basedir}/templates/mining_etc_worker_name dest=/etc/worker_name owner=root group=root mode=0600
    - template: src=${basedir}/templates/mining_etc_worker_password dest=/etc/worker_password owner=root group=root mode=0600
    - copy: backup=no src=${basedir}/files/services/sgminer/run dest=/root/services/sgminer/run owner=root group=root mode=0700

    # - shell: svc -t /service/sgminer

My firsts GPU Rig for mining CryptoCoins (scrypt)

There is many post describing how to make a Windows or a Linux rig to mining crypto coins, so, here, you will just find some useful links, informations or BIOS which helped me.

1. Install Operating System, Dependancy, and xGMINER :

Under Debian Linux :

http://www.lurkmore.com/wiki/Debian_Linux_cryptocoin_mining_rig_setup_guide

Under Windows :

http://www.cryptobadger.com/2013/04/build-a-litecoin-mining-rig-windows/

2. Launch CGMINER / SGMINER with the good options for your cards :

Following a Knowledge database :

https://litecoin.info/Mining_Hardware_Comparison

Find some samples on blog or websites :

http://davixx.fr/blog/cgminer-sgminer-vs-gpu-cards/

dvivgadummy005ah43. Must lie to graphics cards :

It’s a good practice and solve some problem (ADL, or driver loading) to plug into the DVI port, a DVI -> VGA adapter, with some 75 ohms resistors, making that a dummy plug :

http://www.overclock.net/t/384733/the-30-second-dummy-plug

4. Undervolt the GPU vddc :

Using MSI Afterburn, Sapphire TriXX, or an I2C tools knowing technical informations ; or :

By saving the native graphic card BIOS, modifying it with VBE7, load it in the graphic card

http://www.cryptobadger.com/2013/10/undervolting-in-linux-via-modified-bios/

http://rumorscity.com/2013/12/09/how-to-modify-video-bios-to-undervolt-gpu-in-linux/

Using a modified BIOS served by someone else :

http://davixx.fr/blog/gpu-cards-bios-firmware/

Grub2 with GPT and BIOS on Gentoo

Too long time holding Grub2, and using Grub (called now “grub-legacy”).

On a fresh installed Gentoo system, here the links i used to setup it

  1. http://wiki.gentoo.org/wiki/GRUB2
  2. http://ubuntuforums.org/showthread.php?t=1736409
  3. http://www.gentoo.org/doc/fr/grub2-migration.xml

The importants things is that on a BIOS based system (not EFI), and using GPT partitions, you need to give grub a dedicated partition permitting him to store the “second stage” ; (in grub legacy, this code was placed on a kind of unofficial MBR extra-space).

Secondly, i don’t like the way that Grub2 is defaultly provided on linux distributions : /etc/grub.d/* now hold scripts executed in alphabetic order, and their outputs are concatenated to build the configuration. (Note that /etc/defaults/grub also hold some new configuration entries instead keeping theses variables in /etc/grub.d scripts).

To be opposite as myself few lines after, it’s not a bad way to proceed, so let’s go with that :

# kernel-default is not a special kernel name
#  searched by grub2 scripts
# but grub2 scripts search only
#  kernels starting by "kernel-" and "vmlinuz-"

cd /boot
ln -s linux-3.4.56 kernel-default

# partitioning a disk for Grub2 over BIOS/GPT :

# be sure to have an empty GPT partition table :
parted --script --align optimal /dev/sda mklabel gpt
parted --script --align optimal /dev/sda mklabel msdos
parted --script --align optimal /dev/sda mklabel gpt

# our partition needed because of grub2 overs BIOS/GPT
parted --script --align optimal /dev/sda mkpart primary ext2 0% 8MiB

# /boot
parted --script --align optimal /dev/sda mkpart primary ext2 8MiB 192MiB

# swap and root
parted --script --align optimal /dev/sda mkpart primary linux-swap 192MiB 704MiB
parted --script --align optimal /dev/sda mkpart primary ext3 704MiB 100%

# make /boot flagged "boot"
parted --script --align optimal /dev/sda set 2 boot on

# make /dev/sda1 out bios_grub partition
parted --script --align optimal /dev/sda set 1 bios_grub on

# check everything
parted --script --align optimal /dev/sda print


# will install grub files in /boot/grub
grub2-install --grub-setup=/bin/true /dev/sda

# will execute /etc/grub.d scripts
grub2-mkconfig -o /boot/grub/grub.cfg

# writing boot sector 
grub2-install /dev/sda

Hope this will help someone :)

socat permitting MySQL migration

Fast idea using socat (loosed as draft since 2013-02-01) to quickly migrate a tcp and/or unix socket MySQL installation. (socat-1.7.2.0 used for the test).

If the need is temporary, this avoid a MySQL Proxy setup.

Launch that as a daemontools service (for example):

1. /service/socat-mysql-tcp/run

#!/bin/bash
bind_port=3306
bind_addr=127.0.0.1
forward_port=3304
forward_host=sql.dedicated
exec setuidgid mysql socat -d TCP4-LISTEN:${bind_port},bind=${bind_addr},fork TCP4:${forward_host}:${forward_port}

2. /service/socat-mysql-unixsocket/run

#!/bin/bash
sock=/tmp/mysql.sock
forward_port=3304
forward_host=sql.dedicated
exec setuidgid mysql socat UNIX-LISTEN:${sock},fork TCP4:${forward_host}:${forward_port}

gentoo atom regexps (ebuild name and versions)

Working on the Ansible Gentoo Portage emerge module, i’m testing some regexp for matching an atom string.

Atom is a Gentoo Portage pkgcore term.
An atom is a string representing a package name, and, optionally, it version range or version, the ebuild revision, the slot or slot and subslot.

To permit the future ansible module to handle the atom to be a parameter, i have to write some regexp for matching different atom alternative.

The first cases are simply to list, by parsing /usr/portage for avaiables packages, and atom : (here a simple bash/gawk/perl script to check the regexp)

#!/bin/bash

reg_ebuild_without_version='/^([a-z0-9\-]+)\/([a-zA-Z0-9\_\+\-]+)$/'
reg_ebuild_with_version='/^=?([a-z0-9\-]+)\/([a-zA-Z0-9\_\+\-]+)\-([^\-]+(\-r\d+)?)$/'

# this re
find /usr/portage/ -mindepth 2 -maxdepth 2 -type d | \
        gawk -F '/usr/portage/' '{ print $2; }' | \
        perl -pi -e "while(){ next if(${reg_ebuild_without_version}); print; }"

# this regexp must match a app-category/ebuild-name-X.Y.Z-rN
find /usr/portage/ -mindepth 3 -maxdepth 3 -type f -name '*.ebuild' | \
        perl -pi -e 's/\.ebuild$//g;' | \
        gawk -F '/usr/portage/' '{ print $2; }' | \
        gawk -F '/' '{ printf "%s/%s\n",$1,$3 }' | \
        perl -pi -e "while(){ next if(${reg_ebuild_with_version}); print; }"

Thoses regexp are not enougth to match any differents atom.

Some on the missing ones are already testing in the non-finished first version of the emerge module :

davixx/ansible/library/emerge (2013-01-20 version)

I hope, within 7 days, to finish the emerge_parse_atom function for handling successfully  almost 5-6 atom string cases.

ansible – sysctl module

The first version of my sysctl module for ansible have been merged to the devel tree of the mainstream ansible projet (on github) by mpdehaan.

You can now use easily this sysctl module :

  • by using git pull, if you have installed ansible by a simple git clone
  • by downloading manually sysctl in the library subdirectory of your playbooks :
cd /path/to/my/playbooks
[ -d "library" ] || mkdir -v library
wget https://raw.github.com/ansible/ansible/devel/library/sysctl -O library/sysctl
Ansible Servers Orchestration

Ansible is the easiest way to deploy, manage, and orchestrate computer systems you’ve ever seen. You can get started in minutes

After that, you can now use the sysctl module in your playbooks (extracted from the embedded documentation) :

  • Set vm.swappiness to 5 in /etc/sysctl.conf
    (first, check if /proc/sys/vm/swappiness file exist and is writteable, and after change, call sysctl -p and verify if /proc/sys/vm/swappiness file content is now 5) :
sysctl: name=vm.swappiness value=5 state=present
  • Remove kernel.panic entry from /etc/sysctl.conf
sysctl: name=kernel.panic state=absent sysctl_file=/etc/sysctl.conf
  • Set kernel.panic to 3 in /tmp/test_sysctl.conf, check if the sysctl key seems writable, but do not reload sysctl, and do not check kernel value after (not needed, because it is not the real /etc/sysctl.conf updated) :
sysctl: name=kernel.panic value=3 sysctl_file=/tmp/test_sysctl.conf check=before reload=no

gentoo fcron and system crontab

After a new install of gentoo with fcron-3 :

  • there is no /etc/crontab file ; and is it wanted (by default).
  • also, system crontab must not be setup in root crontab’s, but in systab  crontab, using the check_system_crontabs tool.
  • finally, note that by default, /etc/cron.* is not handled by fcron.

So, if you wanna to add support for /etc/cron.* you can :

  • Add this line in a new /etc/crontab (or in an existing if there is no equivalent lines already) :
%hourly * /bin/run-parts /etc/cron.hourly
%daily * * /bin/run-parts /etc/cron.daily
%weekly * * /bin/run-parts /etc/cron.weekly
%monthly * * * /bin/run-parts /etc/cron.monthly
  • Last point do nothing, just a file is updated, not any crontab.
  • Now you call the tool :
check_system_crontabs -v -i -f
  • This command will create or update dynamically the crontab of the systab user (needed by fcron for system crontab, created by the gentoo ebuild)
  • You can after check that the generated systab crontab is the concatenation of your /etc/crontab, any /etc/cron.d/ entry, and a call to the check_system_crontabs tool himself to update – if needed – the crontab – every ten minutes (in fact, for this laspoint, he use the content of /etc/fcron/fcrontab file) :
fcrontab -usystab -l

memcached – List keys in PHP

According to :

There is no official way to list avaiables keys in a memcache process.

The second link show a way, within the debug interface to list them by hundred’s blocks. You can find a modest PHP implementation here :

function getMemcacheKeys($host = '127.0.0.1', $port = 11211){

        $mem = @fsockopen($host, $port);
        if($mem === FALSE) return -1;

        // retrieve distinct slab
        $r = @fwrite($mem, 'stats items' . chr(10));
        if($r === FALSE) return -2;

        $slab = array();
        while( ($l = @fgets($mem, 1024)) !== FALSE){
                // sortie ?
                $l = trim($l);
                if($l=='END') break;

                $m = array();
                // <STAT items:22:evicted_nonzero 0>
                $r = preg_match('/^STAT\sitems\:(\d+)\:/', $l, $m);
                if($r!=1) return -3;
                $a_slab = $m[1];

                if(!array_key_exists($a_slab, $slab)) $slab[$a_slab] = array();
        }

        // recuperer les items
        reset($slab);
        foreach($slab AS $a_slab_key => &$a_slab){
                $r = @fwrite($mem, 'stats cachedump ' . $a_slab_key . ' 100' . chr(10));
                if($r === FALSE) return -4;

                while( ($l = @fgets($mem, 1024)) !== FALSE){
                        // sortie ?
                        $l = trim($l);
                        if($l=='END') break;

                        $m = array();
                        // ITEM 42 [118 b; 1354717302 s]
                        $r = preg_match('/^ITEM\s([^\s]+)\s/', $l, $m);
                        if($r!=1) return -5;
                        $a_key = $m[1];

                        $a_slab[] = $a_key;
                }
        }

        // close
        @fclose($mem);
        unset($mem);

        // transform it;
        $keys = array();
        reset($slab);
        foreach($slab AS &$a_slab){
                reset($a_slab);
                foreach($a_slab AS &$a_key) $keys[] = $a_key;
        }
        unset($slab);

        return $keys;
}