danigm.net - software

rpmlint: Google Summer of Code 2024

danigm — Thu, 30 May 2024 00:00:00 +0200

I'm glad to say that I'll participate again in the GSoC, as mentor. This year we will continue the work done during the past year, as part of the openSUSE project.

So this summer I'll be mentoring an intern and we'll continue working on improving the testing framework of the rpmlint project.

This year we've a better testing framework, thanks to the work done during the past Summer of Code, by Afrid. So the goal for this year is to try to modernize existing tests and remove as much files as possible from test/binary directory, replacing those with mock packages defined with python code.

The selected intern is Luz Marina Montilla Marín. She has done some initial work in the rpmlint project, creating the mock packages for some tests and we've just started with the work to do during the GSoC program, evaluating the tests that we've right now and planning were to start.

She studies at Córdoba, Spain, my hometown. Every year I try to reach young people at different local universities, here in Andalucía, and sometimes I'm able to convince some students to participate, like the GSoC 2020, when Alejandro Dominguez, from Seville, were working on Fractal. So I'm happy that I'm increasing the number of free software developers in my local community :D

I'm sure that she will be able to achieve great things during these three months, so I'm looking forward to start to code and see how far can we go.

rpmlint updates (August 2023)

danigm — Wed, 30 Aug 2023 00:00:00 +0200

We are at the end of the summer and this means that this year Google Summer of code is ending.

The recent changes applied now in the main branch include:

Remove usage of pkg_resource because it's deprecated.
Fix elf binary check with ELF files with a prefix.
New check for python packages with multiple .pyc files for different python versions.
Improve the testing framework (merged the work done during the GSoC 2023)

Summer of Code 2023 updates

The summer of code is ending and the work done by Afrid was good enough to be merged, so I merged it the past week.

I'm really happy with the work done during the GSoC program, now we've a more simple way to define tests for rpmlint checks mocking the rpm, so it's not always needed to build a fake rpm binary for each new test. This will make a lot easier to create simple tests, so I hope that we can increase the code coverage using this new framework.

During this time Afrid has extended the FakePkg class, so it's possible now to define fake metadata and files with fake tags and attributes. It's not complete and it's not a simple task to replace all the rpm binaries used for tests, because the Pkg class and RPM tags is a complex thing, but the current state allow us to replace a lot of them. Afrid has replaced some of the tests that uses binaries, but in the following months we can continue working on this and replace more.

After this work, we can now start to use more the FakePkg class in tests, so another task that we can do is to provide some common fake pkgs to use in different tests and new checks, so now it's possible to create fake packages with dynamic random data, so we can extend tests with fuzz testing and maybe this will help to improve the tool reliability.

Conclusion

I've participated as mentor several times now in the summer of code, and outreachy, and almost always was a good experience. With the gnome foundation in previous programs and this year with opensuse. These two communities are very open to collaboration and makes the whole process really simple, for me as mentor, and also for the intern.

I want to congratulate Afrid, because it was nice to work with him during this summer, he has done a great work, not just technically, but communicating, asking and finding his own solutions without requiring a continuous guidance.

He is very passionate and looks like a nice person, so I hope that he will continue around the open source, it could be opensuse, rpmlint or any other community, but this kind of people is what you want to find in any community.

After many years collaborating with different free software communities, it's amazing that there are so many great people in every project, of course you can find toxic communities and people, but in my experience, that's usually just noise, there are a lot of nice people out there, doing a great work, and I'm happy that young people like Afrid can be part of the free software movement, because this is what makes the free software great, the people that is working on it.

So Thanks a lot to Google for another summer of code, thanks to SUSE for letting me, and encourage me, to mentor, and thanks to all the free software developers that are out there.

I encourage everyone to participate in this kind of programs, for interns, it's a good opportunity to learn and to make some money working on free software, for mentors it's an opportunity to get some help in your project and help newcomers to be part of the community.

Have a lot of fun!

Python in openSUSE Tumbleweed

danigm — Sat, 05 Aug 2023 00:00:00 +0200

openSUSE Tumbleweed is a rolling release distribution, so it's ideal for developers and users that like to have the bleeding edge software.

It's also really "stable" to be a rolling release, from time to time you can find a broken package because of one package is updated and another one is not compatible yet, but it's something that doesn't happen too often thanks to openqa tests, so you don't need to worry about a breaking system.

You can find the Python interpreter and a lot of python modules in every Linux distribution, but Tumbleweed does an interesting thing for Python.

Default Python version (python3 -> python-3.11)

If you don't worry about the python version, you can just rely on python3. In Tumbleweed python3 is not a real package, but the default python version provides python3, so depending on when you install python3 you will get a different package, if you install it today (August 2023) you'll get python311.

In your system you'll have the /usr/bin/python3 binary that points to the default python, so you don't need to worry about the current version, you'll have there the default Python version for the operating system.

In addition to the Python interpreter, you can find in the distribution a lot of python modules, but again you can use the default version, so, for example, if you want to install poetry, you just use zypper install python3-poetry and that will install the real package python311-poetry.

Multiple python versions (3.8, 3.9, 3.10, 3.11)

Besides the default Python, in Tumbleweed you can also find other supported Python interpreters. Right now you can find all the Python versions currently supported by the Python Foundation, from 3.8 to 3.11.

For Python 3.8 you'll only find the interpreter, because the python modules are not built anymore, but for all the other versions you can find almost the same modules.

All python modules that provide binaries uses the update-alternatives, so you can configure in your system the version that you want to use as default. For example, if you want to use the 3.9 version of poetry, having installed different versions you can decide what /usr/bin/poetry points to:

$ sudo zypper in python311-poetry python39-poetry
$ sudo update-alternatives --config poetry

/usr/bin/python3 is a link provided by the default python package, so you can't modify with update-alternatives, so if you want to use a different python version, make sure to do the correct call with the full name python3.9, and use the correct shebang in your python scripts, for example #!/usr/bin/env python3.9.

What happens when default changes?

This way of distributing Python interpreter and modules is useful, because you don't need to update your software to work with the latest Python version, if your software requires another version, you can just install and continue using it, even on a bleeding-edge distribution like Tumbleweed.

But this method has some problems. When the default Python interpreter is changed in the distribution, all packages that depends on python3 will be updated, and that works correctly. But if you have installed some python module using the python3 prefix, that package, and all dependencies, is not updated automatically.

For example, if you installed python3-poetry when python3.10 was the default system, and then the distribution updates the system Python to python3.11, you don't get the python311-poetry package by default, you'll need to install it again. This could break you software, because if you use python3 and the dependencies are not updated, you'll find that some dependencies are not installed after updating.

For that reason, when the default python is changed in Tumbleweed, if you've software that rely on python3 you should make sure to install dependencies again by hand. And also you can do a cleanup and remove all old version packages that you don't need anymore.

If you just want the latest version you can just do all at once with a simple script like this:

#!/bin/bash
# up-py-tw.sh

FROM=$1
TO=$2

if [ $# -lt 2 ]
then
    echo "Usage:   up-py-tw.sh FROM TO"
    echo "example: up-py-tw.sh 310 311"
    exit 0
fi

echo "Updating python packages from $FROM to $TO"

OLDP=$(zypper -q search -i python${FROM}-* | tail --lines +4 | cut --delimiter \| --fields 2)
NEWP=$(echo $OLDP | sed "s/python${FROM}/python${TO}/g")

sudo zypper in $NEWP
sudo zypper rm $OLDP

You should verify what will be installed and what will be removed to make sure that just the python related packages are removed. It's possible that other packages depend on a python version that's older than the system one and that's okay.

Beta version (3.12.0b4)

In Tumbleweed, right now you can find all the supported Python versions, but it's not just that. At this moment you can also find the beta version of the next Python interpreter.

$ sudo zypper in python312

So if you are adventurous and want to test some new feature in the next Python release, you can do it with the version provided there. This version could also be used by openSUSE packagers to test the packages before the Python version is released so it allow us to prepare everything for the release and it could be in the distribution earlier and more tested.

Development

All this multiple python versions is done at distribution level and usually there's only one source package that produces the python39-foo, python310-foo and python311-foo. The source package is usually called python-foo and can be found in the Python devel project.

These packages spec uses the python-rpm-macros to generate all the versions from one source. If you're a packager you can find more information on the openSUSE wiki about Python Packaging.

rpmlint updates (July 2023)

danigm — Thu, 06 Jul 2023 00:00:00 +0200

I'm spending some time every week working in the rpmlint project. The tool is very stable and the functionality is well defined, implemented and tested, so there's no crazy development or a lot of new functionalities, but as in all the software, there are always bugs to solve and things to improve.

The recent changes applied now in the main branch include:

Update the usage of rpm to not use old API.
Fixes for rpmdiff -v, check for NULL char, special macros in comments and spell checking of description in different languages.
Move all the metadata from setup.py to pyproject.toml.
Releasing rpmlint as pre-commit hook
Improvements to the PythonCheck in the dependency checking.

Summer of Code 2023 updates

The first month of the Summer of Code has passed and Afrid is doing a great job there. We've now a draft Pull Request with some initial changes that allow us to mock rpm packages in tests so it's easier to create new tests without the need of creating a binary package.

The first step done was to extend the existing FakePkg class to allow us to define package files and some package metadata.

Now he's working in replacing all of the test_python.py tests that uses binaries rpm to something that doesn't needed.

The idea is to replace as much tests as possible to reduce the number of rpm binaries and after that, provide helper functions, decorators and classes to make it easy to write tests, writing less code.

Roadmap

In any software project there's always room for improvements, fixes and enhancements. If the project is there for enough time, it's even more critical to modernize the code to reduce the technical debt.

My plan for 2023 is to improve the tests around rpmlint as much as possible. First with the GSoC project, making it easier to write more tests, improving the testing tools that we've. And after the summer, improving the test coverage.

There's also a tool that shares some of the ideas with rpmlint, spec-cleaner, it's also written in Python, so the next step, after the tests improvements will be to take a deep look into the code of these two tools and try to integrate in some way. Maybe it's possible to refactor the common code into an external module, maybe we can bring some ideas from spec-cleaner to rpmlint. Not sure yet, but that'll be my next step.

Don't forget that this is free software, so you can participate too! If you find any issue in rpmlint or have an idea to improve it, don't hesitate and create a new issue.

rpmlint: Google Summer of Code 2023

danigm — Tue, 16 May 2023 00:00:00 +0200

I'm glad to say that I'll participate again in the GSoC, as mentor. This year will be a bit different from the previous ones, because I'm not mentoring a GNOME project but a openSUSE project.

I started to work at SUSE the past year and with this new job I get involved in the openSUSE community and I started to contribute to rpmlint.

So this summer I'll be mentoring an intern and we'll work on improving the testing framework of the rpmlint project.

The rpmlint project is a command line tool to check rpm packages, the correctness of these kind of packages and warn the packagers about usual problems or good practices. It's widely used in all Linux distributions based on rpm, mainly SUSE and RedHat.

It is written in Python and uses pytest for testing the code. Right now there are a lot of .rpm binary packages, to check different functionality, but that way of testing makes a bit hard to write new tests and to maintain with changes. The idea of this GSoC project is to extend the testing framework of rpmlint to support an easy way of writting tests that doesn't require a real rpm, something that can mock what it's in the .rpm binary and try to replace some of the current binary tests with this new mock.

The selected intern is Afrid Hussain. He has done some initial work in the rpmlint project, solving some minor issues and we're now preparing the work to be done during the GSoC program. I'm sure that he will be able to achieve great things during these three months, so I'm looking forward to start to code and see how far can we go.

Twitch: GNOME live coding streaming

danigm — Sat, 22 Jan 2022 00:00:00 +0100

This year I've started with something that I wanted to do since the first COVID lockdown, but never did. So as a first year resolution I decided to start streaming and I created my Twitch channel Abentogil.

I've been thinking about streaming since lockdown, when a group of Spanish people created a nice initiative to teach kids how to code and other tech stuff. I never participated on that initiative, but it was the seed of this.

This year I've seen other GNOME streamers doing live coding, like ebassi and Georges and, at the end, I pluck up the courage to start.

The plan

I've started with one our at the end of my day, so I'm trying to stream from Monday to Thursday from 20:00 to 21:00 (CET).

This routing also will help me to spend more time in free software development regularly, so not just on weekends or holidays, but a bit of work everyday.

Right now I've started working on my own projects, like Timetrack and Loop. I've fixed some issues that I had with Gtk4 in Timetrack and I've started to work on the Loop app to add a MIDI keyboard.

The other day I discovered the Music Grid app idea in the gitlab and it's related with the MIDI implementation I've been working on, so the next thing that I'll do on my streams is to create this music grid widget and add it to the Loop app, to have a nice Music creation App.

The language: Live coding in Spanish

I've decided to do the streaming mainly in Spanish. The main reason to do that is because now you can find documentation and a lot of videos in English, but for people that doesn't master the English language, it's harder to follow this content and even harder to participate, to ask or say something.

Spanish is also my main language, and the idea is not just create tutorials or something like that, this is just for fun, and if I'm able to create a small community or influence someone, I want to show that the language is not a unbreakable barrier.

I want to make this for the Spanish community, but that doesn't mean that for some streams I cannot talk in English, all depends on the audience, of course, if a non Spanish speaker comes and ask something, I'll answer in English and try to do multilingual stream.

The International English is the language to use in Free Software, but it's really important to do not convert that in a barrier, because there are a lot of different communities and people that don't know the language and the first contact with something new is always better in your main language. Everything is easier with help from people that's near to your cultural background.

The future

Right now I'm working on Loop and every stream is similar, live coding on this Gtk4 music APP, but in the future I'll do different streams, doing different things on each week day. These are the current ideas that could be a day on my stream:

Maintenance day: Work on my apps, code review, fixes, releases, and development.
GNOME Love day: Give some love to the GNOME project, look for simple bugs to solve in different projects, help with translations, initiatives, and other possible one hour task to give some love to the project in general.
Learning day: Pick some technology and learn by doing, for example, I can explore new programming languages writing a simple app, learn about Linux writing a driver, learn how to animate with blender, anything new to play and discover.
Newcomers day: Create simple tutorial or introduction to some technology, How to write a GNOME Shell extension, how to create a Gtk4 app in python, teach to code from zero, etc.

This is just a list of ideas, if I discover any other interesting use of this time and stream, I can do that. The final goal of this is to have fun, and if someone learns something in the journey, that's even better.

I'll also upload any interesting stream to youtube, so the good content could be viewed on demand.

So if you like this, don't hesitate to view some of the streams and say "Hola":

Loop: A simple music application

danigm — Sat, 11 Dec 2021 00:00:00 +0100

In the last year I've seen some really good musician that performs all the instruments in a song with just a loop machine, recording each instrument one by one in tracks and looping.

I was thinking that it should be easy to have a desktop application that does exactly the same, just some tracks to record some sounds and the playback with a loop option, and that's what I created during this week.

The Loop application is just a simple Gtk4 application that uses gstreamer to record tracks and then you can play each one at the same with or without loop option. With that, a good musician could create the base melody of the song and then sign on top of that. Unfortunately, I'm not a good musician, but I can use this to play around.

I've just created the request on flathub to add the application, so if everything is okay it will be available there soon, so more people can play with this awesome toy.

Right now it has the basic record, play and loop functionality with just four tracks, so don't expect to have a professional music app (yet). The recording and play times are not perfect and there's a delay, that's a known issue, but I'm planning to fix this issue and add more functionality in the future, like:

Make number of tracks configurable
Import track from files
Add a trim slider per track, to be able to adjust the recorder track to loop
Metronome tick and clock to have a visual reference for recording tracks
A record button, but export the combination of all tracks and mic to a mp3 file

And that's all. The logo and design is an initial version done by myself, so if any designer wants to take a look, all contributions are welcome. And of course any code contribution is also welcome.

If you use this application and do some good or fun performance, please, ping me on social networks and let me know.

Endless Orange Week: Hack content creators platform (2)

danigm — Mon, 15 Nov 2021 00:00:00 +0100

The past Friday was the last day of the Endless Orange Week. It was a nice and fun experience, and even if I was not able to do as much as I wanted, we were able to make something that "works" in the Hack project.

The Hack Quest editor

The first step to have custom quests on the Hack app was to complete the Ink language support. We started to work on this some time ago, but never completed the functionality.

I worked on that the first three days, updating the ink library and building the missing pieces to be able to load quests from random paths. I've implemented that in a way that the Hack application is able to receive a path to a .ink file, and it's able to build and run the quets.

The Quests are not just the script, but they have some metadata, like title, subtitle, description, difficulty and the card image to show on the interface. To solve that I defined a "custom quest bundle format", that's bassically a folder with:

questId
quest.jpg
metadata.json
quest.ink

So I also added the functionality to import a bundle zip file and export with the quest information.

I created some command line options to use this new functionality:

  --play-ink=FULL_INK_FILE_PATH                 Start a custom ink quest.
  --import-quest=PATH_TO_BUNDLE_OR_INK_FILE     Import a custom ink quest.
  --export-quest=CUSTOM_QUEST_ID                Export a custom quest bundle.

Quest creation interface and the Inky Editor

The first idea was to try to provide a full quest creation experience in the app, but that was too much, so we decided to simplify the way to create quests and depend on the Inky editor external tool. Manuel Quiñones took some time to update the flatpak application with the latests ink version, so we can use to create custom quests.

The Inky editor provides help about the language, syntax highlighting and a simple way to test the script, so it's a nice tool. The main problem with this tool is that it doesn't provide a way to launch it with a file path so it's not possible to integrate with the Hack app.

So at the end, the Quest creation dialog is just a way to define the metadata and to select the Quest ink files from your filesystem. How the ink script is created is a decision to make for the content creator.

The future

We've no time to complete all we wanted to do, and I didn't create a new release, so this new functionality is still not there. But we'll try to do a release soon.

Simon is working on some interface improvements and also on a new tutorial Quest, so we can introduce the Custom Quest creation tool in the same app.

The Character editor

The other part of this week planning was the character editor. Joana did a really nice work designing the application, the initial assets and the user experience, but I had not too much time to work on the implementation. So I spent just one day working on this.

The main idea was to create a new independent app, and then provide a way to integrate with the Hack application and the custom Quest creation dialog. And it'll be a simple application so maybe it could be useful or interesting for other people, it's a fun way to play around and create random faces.

We just created the application Avatar Creator. I created a simple python Gtk4 application and worked a bit on the basic functionality. So right now it loads a list of svg assets and provide the 3x3 grid. You can click on a grid cell and then choose what basic image should go there.

I added the initial set of basic images, created by Joana, to create this funny robot faces, but the format is simple enough to extend with different "avatar libraries" in the future.

Right now it's also possible to export to png, so the app is functional, but it needs a bit more work.

My idea is to work a bit more in the following weeks, when I have some time, on weekends or holidays and at some point, publish it in flathub. And lets see if there are more developers interested on this app so it can grow.

The application is simple enough to be a good place for GNOME newcomers and it's also a fun project to work on. A simple toy app to create faces that could have some potential, some future ideas:

"Smart" random faces generator
Configurable grid: Maybe is interesting to make it bigger or smaller to play around
Programmed simple base image manipulations, like rotation, mirror, color
Animation creation, maybe be able to export to gif

The Endless Orange Week experience

This week was a really nice experience, because we were working in a "personal" chosen project, that we liked and without the day to day meetings, times schedules and other related work stuff.

But that was not all. In Endless we've different teams that work mostly isolated, because we're working on different fields, we've some overlapping, but we work day to day as small teams, and this week we were all using the same slack channel to show our progress, and it was nice.

Maybe now that we're not a big organization with a lot of workers, we can do something like this more often, it's always good to know more about other coworkers and to learn something that maybe it's not related with your main project, but it could be interesting.

I'm really happy that we did this Endless Orange Week, it's sad that it ended too soon, I'm waiting to learn from my coworkers what amazing things they do during this week and I'm looking forward the next year Orange Week!

Endless Orange Week: Hack content creators platform

danigm — Mon, 08 Nov 2021 00:00:00 +0100

This week (Nov 8 – 12) I am participating in Endless Orange Week, a program where the entire Endless team engages in projects designed to grow our collective learning related to our skills, work and mission.

We propose a project, that could be anything, and then work during the whole week, without distraction. I've choosed to work on the Hack project, that's a really nice project that needs some love, because since the past year, we have other priorities, so there's no time to improve the Hack app.

The project: Hack content creators platform

The Hack application is a "Quest" launcher, and each Quest is an interactive learning experience, a guided introduction to some technology or topic.

Quests are just python scripts, with a simple library to expose messages to the user, ask questions, or wait for desktop events, like launch an application, focus an application, etc. And all these Quests are inside the application, and are created by the Hack team and released with a new Hack app flatpak.

The main idea of the project is to provide a simple Quest editor to allow any Hack user to create and share their our Quests.

To have this Hack content creators platform we'll need:

To simplify the way we create Quests, instead of a python script, we'll uses a Domain Specific Language, called Ink. We started to work on this, but we never ended the support.
To create the interface to be able to import and export custom Quests, that could be zip bundles, with the Ink script and some images.
To create the interface to write the actual Quests and save or bundle.
Create some introductory Quest to explain "how to create your own Quests!".
Create an character editor, to be able to "design" new characters for Quests.

The Quest editor

The Quests will be written using the Ink language, and there's something done before. The first idea is to just provide a text editor and some helpful information about the format, and maybe a button to validate. But if there's time we can use something advanced or even integrate the Inky editor.

The Character editor

Each Quest has a main character, and we've five in the Hack app right now, but it could be great to be able to define new ones for custom Quests. That's the idea of this part of the project.

The initial idea is to have a library of character parts to combine, and the editor will allow the user to combine this parts and maybe change colors, to be able to create unique characters for your Quests.

The Team

I'm not working alone in this "side" project during the Endless Orange Week, Simon Schampijer and Joana Filizola will be working on this too, so this is a big task but we've a great team. Let's see how far we are able to go during just one week.

The impact

This is just a project to try to keep alive more time the Hack application, without a lot of effort or a whole team behind it. We are not able to put more content there periodically, so if there's a way to create new content easily and (maybe in the future) a way to publish, it'll be possible to create a community around the project.

And we have also new possibilities, in the near future, we can add some Hack content to the Endless Key, and using the Ink language, so this editor could help to bring more content there easily.

And the final piece, the Character editor, could be an independent application, a nice simple application that could be used to create your character for your profile photo, or to generate random character pics.

Fractal: Refactoring and the review process

danigm — Tue, 30 Jun 2020 00:00:00 +0200

In this year GSoC, Alejandro is working on Fractal, moving code from the backend to the client, to try to simplify the code used to communicate with the matrix.org server and maybe in the future we can replace fractal-matrix-api with the matrix-rust-sdk. And then we'll have less code in our project to maintain.

This is a great work, something needed in a project with a technological debt of several years. I created this project to learn Rust, and also I was learning about the matrix protocol during the project build. And other contributors do the same. So we've been building one thing on top another for a lot of years.

In this kind of community driven projects it's the way to go. For some time we've people interested and developers think about the design and start change some parts or to write new functionality following a new design pattern. But voluntary developers motivation change in time and they left the project and the next one continues the work with a different vision.

It's not something bad, it's the greatness of the open source. Different people has different motivations to participate in a free software project, and every contribution is welcome. I'm the maintainer of the project and I've spent a lot of time building Fractal, but I don't have the same motivation now to work on the project, to it's good to have other people working on it so it can continue alive.

Alejandro is doing a great work and he's not a 4 months contributor. He's working on the backend refactoring for 2 years now, step by step and he has plans for the future.

Refactoring a big project is always hard, because there's a lot of code movement and always there's the fear to regressions.

Rust is a great language and it shines when big code refactoring comes to the scene. If it compiles, you know that there's no memory errors, dangling pointers and that kind of problems. If it builds it will work.

But maybe it will work different, so the review process is needed to ensure that the application continues working.

Automated tests are really useful for big code changes and project refactoring, because you have a quick picture and some certainty that the project is working. But we don't have tests in fractal :D, so someone should do that.

So here I am. Reviewing large MR, with a lot of lines. At least gitlab makes this process a bit easier.

What I'm trying to do in the review process is to just read the whole diff and check if there's some problem in the code. And after every change, I run the app and I do some tests, trying to use the functionality that could be broken by those new changes.

This takes a lot of time and it's not something fun to do... But someone has to do that. And during that process, sometimes I learn something new. Reading code is an interesting task and try to find bugs in code while reading it, is something useful. To think about that code, what it does and why.

Fractal: Google Summer of Code 2020

danigm — Fri, 08 May 2020 00:00:00 +0200

I'm glad to say that I'll participate again in the GSoC, as mentor. This summer we'll try to implement multi-account support in Fractal.

The selected student is Alejandro Dominguez (aledomu), that is collaborating with Fractal since the Seville Hackfest in December 2018, doing a great work in the backend. Alejandro is young developer with a lot of energy and ideas, so it's great to have this kind of people working on GNOME.

I've not spend much time lately developing Fractal, the time and energy is limited, but I'll try to use this GSoC mentorship to go back to the active Fractal development. My objective during this summer will be to stabilize, fix bugs and improve the performance and create a new release at the end of the GSoC, because we've a lot of new functionality in master, but I didn't spend the time to do the release.

The google summer of code is a great opportunity for free software, it give us a full time working student, during three months, working on free software, so it's really appreciated in a community where a lot of work is volunteer work.

There are a total of 14 projects selected for this GSoC round for gnome, you can take a look to the full list in the GSoC page.

Andaluh-rs, a lib to transcript Spanish to Andaluh

danigm — Sat, 06 Jul 2019 00:00:00 +0200

Spain is a big region with a lot of languages, we've the official one for the whole region, Spanish or Castilian (es-ES), and other officials languages spoken in other regions, like the Galician (gl-ES) spoken in Galicia, the Basque (eu-ES), spoken in the Basque Country, the Catalan (ca-ES) spoken in Catalonia.

But we've more languages that doesn't have the same official support, the Valencian, the Aragonese, the Asturian, and more. And also we've some dialects like the Andalusian.

All of this languages was discredit during the dictatorship period when the only official language was the Spanish and the others was treated like vulgar languages or non educated one, and in some cases the language was prohibited.

Whe the democracy arrives, some regions spend a lot of resources trying to recover the language and are supporting it until now with official institutions and lessons in the official education system. But in other regions the stigma continued until today.

That's the case of the Andalusian, that right now is not considered a language but a dialect. In any case, this dialect is treated as a non cultivated language, spoken by illiterates. We've a lot of Spanish movies and Series where the character that spoke Andalusian was the illiterate, from the countryside or the servant of the family.

Be proud of it

In Andalusia we've a lot of culture, literature and music done in Andalusian, but we don't have a way to write that and many people try to avoid the acent and the local words to look more cultured and because there's no a writing system, we write in Spanish, it's hard to write lyrics, poetry and other kind of literature. The Andalusian is not only a way to talk, some words are short and we've contractions and other vowel sounds so if you write in Spanish, it's not the same as spoken, some information is missed and for example in music or poetry the metric doesn't match.

There's a movement trying to define a writing Andalusian and promoting the language, trying to make people proud of it and talk and write without complexes.

Translator, here comes the code

And there's a group of developers that are working in some tools to provide direct translation from Spanish and other tools to ease the Andalusian writing.

I like to write code and I'm always happy to find new problems to solve, to learn new languages, tools and to spend some time trying to code something that I've not done before. So I decided to write a translator from Spanish to Andaluh using rust, and I've created the andaluh-rs lib.

The translator is more or less easy, there're some rules that should be applied from top to bottom that basically replaces some group of letters. There's a implementation in python that uses regular expressions for that. There're a lot of regular expressions, so I thougth that it could be easy to use a parser, so I used the pest parser.

// supress muted /h/

H = { ("h" | "H") }
initial_h = { H ~ letter }
CH = { C ~ H }
inner_ch = { CH ~ letter }
inner_h = { !inner_ch ~ H ~ letter }
hua = { H ~ ("ua" | "UA" | "Ua" | "uA") }
hue = { H ~ ("ue" | "UE" | "Ue" | "uE") }
noh = { !CH ~ !H ~ letter }
h = _{ ((sp|SOI)? ~ initial_h* ~ ((hua | hue | inner_ch | inner_h) | noh+)+)+ }

I've defined each rule in the pest format, so I've a parser for each rule and then I can replace the word with the correct replacement.

pub fn h_rule(input: &str) -> Result<String, Error> {
    rule!(Rule::h, &input, Some(&defs::H_RULES_EXCEPT),
        Rule::initial_h | Rule::inner_h => |pair: Pair<Rule>| {
            let s = pair.as_str();
            let h = slice!(s, 0, 1);
            let next = slice!(s, 1);
            keep_case(&next, &h)
        },
        Rule::hue => |pair: Pair<Rule>| {
            keep_case("güe", &pair.as_str())
        },
        Rule::hua => |pair: Pair<Rule>| {
            keep_case("gua", &pair.as_str())
        })
}

To simplify the code, I've defined the rule macro, with the code used in all rules:

macro_rules! rule {
    ($rule: expr, $input: expr, $( $($t: pat)|* => $r: expr ),* ) => {{
        let map: Option<HashMap<&str, &str>> = None;
        rule!($rule, $input, map, $( $($t)|* => $r ),*)
    }};
    ($rule: expr, $input: expr, $map: expr, $( $($t: pat)|* => $r: expr ),* ) => {{
        let (repl, input) = match $map {
            Some(ref m) => replace_exceptions($input, m),
            None => (vec![], $input.to_string())
        };

        let pairs = AndaluhParser::parse($rule, &input)?;
        let mut output: Vec<String> = vec![];

        for pair in pairs {
            let chunk = match pair.as_rule() {
                $( $($t)|* => {
                    $r(pair)
                } ),*
                _ => {
                    String::from(pair.as_str())
                },
            };
            output.push(chunk);
        }

        let mut outstr = output.join("");

        if $map.is_some() {
            outstr = replace_exceptions_back(&outstr, repl);
        }

        Ok(outstr)
    }}
}

And because the Spanish and the Andaluh languages uses unicode and rust Strings can not be iterated by unicode, I've used unicode_segmentation crate, and defined some utility macros to get the real String len and to get slices of that String.

macro_rules! chars {
    ($input: expr) => {
        UnicodeSegmentation::graphemes($input, true)
    }
}

macro_rules! slice {
    ($input: expr, $start: expr, $end: expr) => {
        chars!($input)
            .skip($start)
            .take($end - $start)
            .collect::<String>()
    };
    ($input: expr, $start: expr) => {
        chars!($input)
            .skip($start)
            .collect::<String>()
    }
}

macro_rules! len {
    ($input: expr) => {
        chars!($input).count()
    }
}

With all this done, we only have to apply all rules, in the correct order, to the input string so we can get the translated String as output.

pub fn epa(input: &str) -> Result<String, Error> {
    // TODO: escape links
    let rules = [
        h_rule,
        x_rule,
        ch_rule,
        gj_rule,
        v_rule,
        ll_rule,
        l_rule,
        psico_rule,
        vaf_rule,
        word_ending_rule,
        digraph_rule,
        exception_rule,
        word_interaction_rule,
    ];

    let mut output = input.to_string();
    for r in rules.iter() {
        let out = r(&output)?;
        output = out.to_string();
    }

    Ok(output)
}

Performance

This code is not the best one, I'm doing a lot of string operations with copies and clones, I'm sure that anyone with more experience with rust can view a lot of points where we can optimize this code. At first I thought that the translation could be done during the parsing, keeping a length to be able to view backward and forward.

Maybe it's possible to read char by char, keeping a buffer, and detecting if we can apply any of the rules with the content in the buffer, but I've based this lib in the python one, so for me it was easier to translate each regex to pest regex and then do the same translations in the lib.

But I still think that there's a better solution for this problem, but some times it's better to have something that just works instead of a never done best solution.

During this process I've learned to use pest and I've been playing a lot with regular expressions, so it was a fun project.

Kung-fu Master, Blindfolded debugging

danigm — Sun, 05 May 2019 00:00:00 +0200

Recently I've been working in the EOS update. The change is really big because EOS has some modifications over the upstream code so there are a lot of commits applied after the last stable upstream commit.

EOS 3.5 was based on gnome 3.26 and we are updating to use the gnome 3.32 so all the downstream changes done since the last release should be rebased on top of the new code and during this process we refactor the code and commits using new tools and remove what's now in gnome upstream.

I've been working in the Hack computer custom functionality that's on top of the EOS desktop and basically I've been rebasing the code in the shell to do the Flip to Hack, the Clubhouse, a side component and notification override to propse hack quests and the wobbly windows effects.

I've been working mainly with gnome shell and updating javascript code to the new gjs version, but this was a big change and some rare bugs.

Here comes the blindfolded debugging. I've a development system with a lot of changes and a functionality in the gnome shell that depends on different projects, applications and technologies.

I don't know the code a lot, because I'm not the one that wrote all of this, I'm working here since February, so I know a bit how things works, but I'm not the one who knows everything, there are a lot of rare cases that I don't know about.

I've found and fixes several bugs in different projects, that I don't know about a lot, during this process. How can I do that? If you are a developer that write code since a few years maybe you've experienced something similar, I'm calling this the blindfolded debugging technique:

Start to change code without knowing exactly what you're doing, but with a small feeling that maybe that line is the problem.

This technique is only for experienced programmers, as a kungfu master that put a blindfold in their eyes to fight against an opponent, the developer that will be brave enough to try this should have a lot of experience or he will fail.

You're an experienced developer, but you don't know the software that you're debugging. It doesn't matter. The same way that in a kungfu fight you don't know your opponent, but almost every fighter has two arms and two legs, so more or less you'll know that he'll try to punch you or kick you, you've a clue. As a programmer, every software has an structure, functions, loops...

No matters who wrote that or how old that code is, if you're an experienced programmer you'll feel the code and without knowing exactly why or how, you will be able to look at one line and says: Here you're little buggy friend.

Maybe I'm only trying to justify my lucky with a story to feel better and say that I'm not a dumb changing random lines to try to find a bug, but I'm sure that other developers has this feeling too, that feeling that guides you to the exact problem but that you're not able to rationalize.

I think that this is the experience, the expertise is your body and your inner brain doing the work meanwhile you don't need to think about it or know exactly what you are doing.