danigm.net - python

openSUSE: The new git workflow

danigm — Fri, 14 Nov 2025 12:00:00 +0100

openSUSE is migrating the package source management from osc to git. I will try to explain here what it is the "git workflow" in openSUSE and give some practical information for contributors and maintainers.

You can find some documentation in the openSUSE wiki

Why?

openSUSE uses Open Build Service to store package sources (.spec files and patches). The source control is similar to subversion, so you can osc checkout, osc commit, osc log, etc. to work with any package source.

I don't know all the reasons to do the change, and different people will find different reasons, to support the change or to be against it. I will just write here what I consider a good reason to migrate:

Almost everyone has moved to git now, so today, git is the default source code management (scm) tool, almost all the people knows about github, so it's good to be "standard".
With the concept of "cheap" branches in git, it could be easier to maintain different versions of the same package, same repository, different branches, instead of actual forks.
Decouple the package source management from build. OBS does a lot of things, moving source code and review process to gitea could reduce the complexity (but requires some integration).

From OBS to gitea

All the development happens in OBS, so the classic workflow happens entirely there:

developer: branch -> checkout -> modify -> commit -> submit request
maintainer: review -> accept/decline

And everything is integrated in OBS that does the corresponding build of the sources, checks with services, forward, etc.

The new workflow changes almost all the interaction from OBS to gitea. OBS is still a really important piece of software in the process, but in the new workflow it's a build backend, so the user interaction will be with the git repo in gitea.

The new git workflow happens in gitea:

developer: fork -> clone -> modify -> commit -> push -> pull request
maintainer: review -> approve/decline

As you may notice, the "default" workflow is very similar, the difference is in the tools, the classic workflow occurs in build.opensuse.org and the new workflow in src.opensuse.org. And the tools to work with the sources are also different, osc in the first one and git in the new workflow.

And with this new workflow, there are some bots in gitea that sends the changes to OBS, so for any pull request created you will have the build results and approval from the bot if everything is building.

How to modify a package (leap 16.0)

Notice that you should have a openSUSE account to work with the gitea instance, and don't forget to configure your ssh key to be able to push using the gitea@src.opensuse.org remote.

So as a contributor, this is the basic workflow to follow to update a package in openSUSE:

Look for the source package in the pool (https://src.opensuse.org/pool)
Fork it in your user space, click the top right button. Skip this step if you have forked before
Clone your fork and work on that repo, create a new branch, modify the spec file, add new soures, etc. And make sure to update the .changes file accordingly, you can still use osc vc to do that.
You can build locally your package with osc build, as usual, but you will need to specify the build project:

$ git obs meta pull
$ git obs meta set --project openSUSE:Backports:SLE-16.0
$ osc build

Once you are happy with your changes, you can commit, push to your fork and then you can create a Pull Request. The pull request should target the pool repo and desired product branch. Right now you can just target leap-16.0, as factory is not migrated yet.

Devel projects

Some devel projects have been migrated now to git, so a similar workflow is available to modify these packages in Tumbleweed. The development of these packages is not happening in the pool, so you need to find first the devel project.

There's no simple way to discover the actual package devel source, as far as I know the easier way is:

Look for the devel project of the package in the Factory list, (ex python-pytest)
Go to the devel project in OBS (ex devel:languages:python:pytest)
Click on the link This project is managed in SCM

Once you have located the desired package to modify, the workflow is similar to what I explained before, but instead of forking from pool, you should fork from the devel project. For example if you want to modify python-pytest you should fork from https://src.opensuse.org/python-pytest/python-pytest, and create the pull request for that repo, to the main branch.

Adding a new package follows a different process that's documented in the wiki

What's migrated?

As I said, the migration is happening right now, so not everything is migrated. The first project migrated was the Leap 16.0. And we are slowly migrating some devel projects.

Right now from the python-maintainers team, we've started the migration of two subprojects:

You can find more information about the migration current state in the wiki: https://en.opensuse.org/opensuse:obs_to_git#codestream_project_status_table

Python 3.13 Beta 1

danigm — Wed, 22 May 2024 00:00:00 +0200

Python 3.13 beta 1 is out, and I've been working on the openSUSE Tumbleweed package to get it ready for the release.

Installing python 3.13 beta 1 in Tumbleweed

If you are adventurous enough to want to test the python 3.13 and you are using openSUSE Tumbleweed, you can give it a try and install the current devel package:

# zypper addrepo -p 1000 https://download.opensuse.org/repositories/devel:languages:python:Factory/openSUSE_Tumbleweed/devel:languages:python:Factory.repo
# zypper refresh
# zypper install python313

What's new in Python 3.13

Python interpreter is pretty stable nowadays and it doesn't change too much to keep code compatible between versions, so if you are writing modern Python, your code should continue working whit this new version. But it's actively developed and new versions have cool new functionalities.

New and improved interactive interpreter, colorized prompts, multiline editing with history preservation, interactive help with F1, history browsing with F2, paste mode with F3.
A set of performance improvements.
Removal of many deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib, lib2to3.

Enabling Experimental JIT Compiler

The python 3.13 version will arrive with an experimental functionality to improve performance. We're building with the --enable-experimental-jit=yes-off so it's disabled by default but it can be enabled with a virtualenv before launching:

$ PYTHON_JIT=1 python3.13

Free-threaded CPython

The python 3.13 has another build option to disable the Global Interpreter Lock (--disable-gil), but we're not enabling it because in this case it's not possible to keep the same behavior. Building with disabled-gil will break compatibility.

In any case, maybe it's interesting to be able to provide another version of the interpreter with the GIL disabled, for specific cases where the performance is something critical, but that's something to evaluate.

We can think about having a python313-nogil package, but it's not something trivial to be able to have python313 and python313-nogil at the same time in the same system installation, so I'm not planning to work on that for now.

Where's my python code?

danigm — Sat, 03 Feb 2024 00:00:00 +0100

Python is a interpreted language, so the python code are just text files with the .py extension. For simple scripts it's really easy to have your files located, but when you starts to use dependencies and different projects with different requirements the thing starts to get more complex.

PYTHONPATH

The Python interpreter uses a list of paths to try to locate python modules, for example this is what you can get in a modern GNU/Linux distribution by default:

Python 3.11.7 (main, Dec 15 2023, 10:49:17) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['',
 '/usr/lib64/python311.zip',
 '/usr/lib64/python3.11',
 '/usr/lib64/python3.11/lib-dynload',
 '/usr/lib64/python3.11/site-packages',
 '/usr/lib64/python3.11/_import_failed',
 '/usr/lib/python3.11/site-packages']

These are the default paths where the python modules are installed. If you install any python module using your linux packaging tool, the python code will be placed inside the site-packages folder.

So system installed python modules can be located in:

/usr/lib/python3.11/site-packages for modules that are architecture independent (pure python, all .py files)
/usr/lib64/python3.11/site-packages for modules that depends on the arquitecture, that's something that uses low level libraries and needs to build so there are some .so files.

pip

When you need a new python dependency you can try to install from your GNU/Linux distribution using the default package manager like zypper, dnf or apt, and those python files will be placed in the system paths that you can see above.

But distributions doesn't pack all the python modules and even if they do, you can require an specific version that's different from the one packaged in your favourite distribution, so in python it's common to install dependencies from the Python Package Index (PyPI).

Python has a tool to install and manage Python packages that looks for desired python modules in PyPI.

You can install new dependencies with pip just like:

$ pip install django

And that command looks for the django python module in the PyPI, downloads and install it, in your user $HOME/.local/lib/python3.11/site-packages folder if you use --user, or in a global system path like /usr/local/lib or /usr/lib if you run pip as root.

But the usage of pip directly in the system is something not recommended today, and even it's disabled in some distributions, like openSUSE Tumbleweed.

[danigm@localhost ~] $ pip install django
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try
    zypper install python311-xyz, where xyz is the package
    you are trying to install.

    If you wish to install a non-rpm packaged Python package,
    create a virtual environment using python3.11 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.

    If you wish to install a non-rpm packaged Python application,
    it may be easiest to use `pipx install xyz`, which will manage a
    virtual environment for you. Install pipx via `zypper install python311-pipx` .

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

virtualenvs

Following the current recommendation, the correct way of installing third party python modules is to use virtualenvs.

The virtualenvs are just specific folders where you install your python modules and some scripts that make's easy to use it in combination with your system libraries so you don't need to modify the PYTHONPATH manually.

So if you've a custom project and want to install python modules you can create your own virtualenv and use pip to install dependencies there:

[danigm@localhost tmp] $ python3 -m venv myenv
[danigm@localhost tmp] $ . ./myenv/bin/activate
(myenv) [danigm@localhost tmp] $ pip install django
Collecting django
...
Successfully installed asgiref-3.7.2 django-5.0.1 sqlparse-0.4.4

So all dependencies are installed in my new virtualenv folder and if I use the python from the virtualenv it's using those paths, so all the modules installed there are usable inside that virtualenv:

(myenv) [danigm@localhost tmp] $ ls myenv/lib/python3.11/site-packages/django/
apps  contrib  db        forms  __init__.py  middleware   shortcuts.py  templatetags  urls   views
conf  core     dispatch  http   __main__.py  __pycache__  template      test          utils
(myenv) [danigm@localhost tmp] $ python3 -c "import django; print(django.__version__)"
5.0.1
(myenv) [danigm@localhost tmp] $ deactivate

With virtualenvs you can have multiple python projects, with different dependencies, isolated, so you use different dependencies when you activate your desired virtualenv:

activate $ . ./myenv/bin/activate
deactivate $ deactivate

High level tools to handle virtualenvs

The venv module is a default Python module and as you can see above, it's really simple to use, but there are some tools that provides some tooling around it, to make it easy for you, so usually you don't need to use venv directly.

pipx

For final python tools, that you are not going to use as dependencies in your python code, the recommended tool to use is pipx.

The tool creates virtualenv automatically and links the binaries so you don't need to worry about anything, just use as a way to install third party python applications and update/uninstall using it. The pipx won't mess your system libraries and each installation will use a different virtualenv, so even tools with incompatible dependencies will work nicely together in the same system.

Libraries, for Python developers

In the case of Python developers, when you need to manage dependencies for your project, there are a lot of nice high level tools for managing dependencies.

These tools provides different ways of managing dependencies, but all of them relies in the use of venv, creating the virtualenv in different locations and providing tools to enable/disable and manage dependencies inside those virtualenvs.

For example, poetry creates virtualenvs by default inside the .cache folder, in my case I can find all poetry created virtualenvs in:

/home/danigm/.cache/pypoetry/virtualenvs/

Most of these tools add other utilities on top of the dependency management. Just for installing python modules easily you can always use default venv and pip modules, but for more complex projects it's worth to investigate high level tools, because it'll make easy to manage your project dependencies and virtualenvs.

Conclusion

There are a lot of python code inside any modern Linux distribution and if you're a python developer it's possible to have a lot of python code. Make sure to know the source of your modules and do not mix different environments to avoid future headaches.

As a final trick, if you don't know where's the actual code of some python module in your running python script, you can always ask:

>>> import django
>>> django.__file__
'/tmp/myenv/lib64/python3.11/site-packages/django/__init__.py'

This could be even more complicated if you start to use containers and different python versions, so keep you dependencies clean and up to date and make sue that you know where is your Python code.

Hackweek 2023

danigm — Tue, 03 Jan 2023 00:00:00 +0100

Hack Week is the time SUSE employees experiment, innovate & learn interruption-free for a whole week! Across teams or alone, but always without limits.

This year the Hack Week was this week, the last week of January and for my first SUSE hack week I decided to work in something funny, LILS.

Linux Immersive Learning System (LILS)

I don't think that this is a good name, but don't focus on it. The main idea of this project is to create some basic machinery to be able to write "interactive" tutorials or games using the INK language.

This is not an original idea, indeed all I've done is something that's currently working on EndlessOS, and was the main idea behind the dead project Hack Computer, you can even take a look to the Hack app in flathub. But I wanted to work around this, and create something simpler, from scratch.

I wanted to build something simple, with just Python, and make it simple enough to be able to build other tools on top. The design is simple, an INK parser, with a simple game runner. In the INK script you can define commands, to do something special, and wait for events with listeners, to wait for an event in the OS to continue.

With this basic functionality it's possible to build different user interfaces for different environments. And the original idea was to make the commands and listeners something extensible with a simple API, but that's something that I have not done yet, it's all Python functions without extension point.

The code can be found in github.

The INK parser

The most complex part of this project is the INK language parser. The Ink parser is free software and there's a Linux version that you can use to parse and compile to json, but I wanted to create my own parser with Python.

I've spent most of the Hack Week time fighting with the parser and indeed was the most challenging and fun part, because I've not worked a lot with parsers and it's not something easy as pie 😛️.

I remember creating a java compiler long time ago, when I was in the Seville University, for the Language Processors course. We did that with ANTLR, so starting from that, and looking for a Python lib, I found the Lark project. So if you like regular expressions, writing a grammar is a lot more FUN.

At the end I was able to support some basic INK language with support for:

Text
Tag support
Options, with suppress text support
Knots, Stitches and Diverts
Include other .ink files
Variable definition and basic operations
Knots and Stitches automatic visiting count variables
Conditional options using variables

It still fails in some cases, the comments and TODO placed in between text is not detected correctly and there's a lot of complex stuff that's not supported yet, but with what's supported right now it's possible to create complex scripts with loops and complex game graphs, so it's good enough to build games just with it.

GNOME shell extension

To integrate with the system I've done a simple GNOME shell extension. The extension just shows the text as bubbles and options as buttons, it's really simple and I've no time to make it something ready to be used, but I was able to make something usable.

To be able to run the LILS python library from gjs I've created a simple dbus service that exposes the basic InkScript class functionality as a dbus API.

I was thinking about being able to change the desktop background, depending of the value of a background variable in the script and do something similar to play music and sounds, so it could be a cool game engine with some additions.

SUSE Hack Week

So this Hack week was really fun and I learned a lot. It's really great that SUSE does things like this, letting us work in different projects for a week, to learn, to grow or to just explore different paths.

Timetrack app for GNOME

danigm — Sat, 26 Jan 2019 00:00:00 +0100

This week I started a new small project called Timetrack, you can find the code in the gnome gitlab.

This is a simple app to store an activity log in a sqlite database and to provide useful reports in the future. The main idea is to use this for may day to day work, so I'll be able to keep track of my working hours and other activities.

The need

There's a lot of timetrack applications. There are good web apps and also some desktop applications. At first, I tried to find a gnome-shell plugin, but I didn't find any extension that is good for me.

I've had a custom simple web app to track my time.

I've been using this app for a long time, but I don't like to use web apps for simple tasks so I want something local.

So I tried with some gtk desktop applications, but none of the applications that I've found looks good to me.

I wanted a simple timetrack app, without any complex stuff, so I thought that it won't be hard to implement it. So I started with a simple python app based on the code of Password Safe.

So I started to code and in two days I had a simple application that works. The applications is now on flathub so it's available for use.

The functionality

Right now, Timetrack has a limited set of features, but I think I'll improve the app with new features as soon as I need something. So, the current version can:

Track activity time with a simple button, showing the activity time spent
List last activities
Edit / Delete activities
Simple report by day, week or month
Report navigation using a calendar

I've plans to add more functionalities in the near future, like:

Report export to plain text and csv
Comments for activities
Tags for activities
Detailed reports
Activity Graphs

All of this is stored in a sqlite database, if you install the app using flatpak you can find the database in this path: ~/.var/app/net.danigm.timetrack/data/timetrack.sqlite3.

I've done the icon for this app using the gnome-clocks icon and adding some colors to it.

If you find this app useful, don't hesitate and use it. Any feature request or bug report is welcome. And of course, all is free software, so if you want to collaborate, go ahead and send me some Merge Request.

Monkey Patching

danigm — Thu, 02 Feb 2017 00:00:00 +0100

Qué es el Monkey Patching

El Monkey Patching es una técnica de programación de los lenguajes dinámicos que consiste en modificar el código en tiempo de ejecución.

Esto quiere decir que en lugar de modificar el código fuente de una clase, por ejemplo, en tiempo de ejecución asignamos otra función a ese método y todas las llamadas posteriores en lugar de ejecutar el código definido en la clase ejecutarán el código "parcheado".

Esto está relacionado con el llamado Duck typing, que viene a decir que si nada como un pato y suena como un pato, esto es un pato. Y esto del pato hace referencia a que los tipos no tienen que cumplir una interfaz estricta, simplemente, si se comportan como tal, es el tipo esperado. Y con el Monkey Patching podemos hacer que un perro haga cuak como un pato, modificando el objeto perro en tiempo de ejecución.

If it walks like a duck and talks like a duck, it’s a duck, right? So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect.

-- Patrick Ewing

Qué hay que tener en cuenta para el Monkey Patching

Para poder hacer Monkey Patching necesitamos, tipado dinámico, ya que si un método espera un objeto tipo pato y le pasamos uno tipo perro, pero "parcheado", el método no debe quejarse.

También tenemos que tener en cuenta la vida útil de los módulos y objetos. Si parcheamos un módulo, es necesario conocer el alcance de esas modificaciones, si sólo afectarán a mi módulo o si pueden afectar a otros módulos que lo importen, o si parcheamos una instancia, es necesario conocer hasta dónde llegarán esas modificaciones.

Y por supuesto también es necesario tener acceso a las partes del código que se quieran modificar, si vamos a modificar un objeto con métodos privados.

Por suerte, en Python no tenemos métodos privados realmente, se suele usar el guión bajo (_metodo), que según la herramienta lo oculta como privado, pero en realidad en Python todo es accesible y por tanto podemos parchear lo que queramos.

Algunos ejemplos

Veamos el ejemplo que viene en la wikipedia:

>>> import math
>>> math.pi
3.141592653589793
>>> math.pi = 3
>>> math.pi
3

En este ejemplo se ve que se importa el módulo math y se modifica el valor de math.pi. Fácil y sencillo. No tiene mucho sentido este cambio, pero si lo hacemos en nuestro código, ya todo el código que venga después y use math.pi recibirá como valor 3, y esto afecta incluso a librerías de terceros.

Algo más práctico con código Django:

from django import shortcuts

old_render = shortcuts.render

def custom_render(*args, **kwargs):
    t1 = time.time()
    resp = old_render(*args, **kwargs)
    s = time.time() - t1
    print("Render time: %s seconds" % s)
    return resp

shortcuts.render = custom_render

En este ejemplo modificamos el comportamiento por defecto de la función render de django. Reemplazamos esta función por otra que lo único que hace es medir el tiempo que se tarda en la llamada a la función inicial y sacarlo por pantalla.

Otro ejemplo con Django:

def password_logger(self, newp):
    send_to_hacker_email(self.username + ":" + newp)
    self.real_set_password(newp)

user = User.objects.get(username="admin")
user.real_set_password = user.set_password
user.set_password = password_logger

Aquí se muestra cómo se puede modificar una instancia. En este ejemplo se modifica el método set_password del usuario admin, para que cuando se llame a este método se envíe la nueva contraseña por email a un presunto hacker.

En el caso de los modelos de django no tiene mucho sentido modificar la instancia, mejor sería modificar el modelo, la definición de la clase, pero vale como ejemplo.

Cuándo usar el Monkey Patching

El Monkey Patching es una herramienta muy poderosa, pero a la vez, es algo tremendamente peligroso, ya que hace que una clase o un módulo no se ejecuten como se espera, como dice su definición y por tanto puede dar más de un dolor de cabeza.

Un gran poder conlleva una gran responsabilidad

-- Ben Parker

Por esto hay que saber cuándo usar el Monkey Patching y cuando no.

Cuando SÍ:

Depuración o ejecución paso a paso: pdb, ipdb, mientras depuramos un código para que nos saque información por pantalla, etc.
Testing: Para reemplazar métodos, atributos o funciones en tiempo de ejecución en los tests, por ejemplo una función que genera números aleatorios la podemos transformar en algo predecible para testear otros métodos relacionados, o podemos reemplazar funciones que tarden mucho tiempo debido a sistemas externos, para que los tests se puedan ejecutar rápidamente.
Fixs o arreglos de libs externas: Para aplicar un parche o un arreglo de una biblioteca externa en una versión que aún no ha sido parcheada oficialmente.

Cuando NO:

En la mayoría de las ocasiones, no es recomendable usar Monkey Patching. Por lo tanto, si hay otra forma de hacer lo que queremos hacer, seguramente sea mejor idea que el Monkey Patching.

Como he comentado antes, el Monkey Patching hace que el código que se ejecuta sea diferente del descrito en la definición de la clase o módulo, por tanto, hay que evitarlo y cuando se use hay que documentarlo muy bien, porque si no, puede enmascarar problemas o hacer que futuras modificaciones al código original no tengan efecto.

Resumiendo, el Monkey Patching está muy guay, pero úsalo bajo tu propia responsabilidad.