# Projects

- URL: https://giubots.net/projects/

## Multimodal Conversational Toolkit (MCTK-TINI)

The work of [MMCC](#multimodal-chatbot-creator-mmcc) was generalised in my Master's thesis and turned into a multimodal conversational toolkit, [MCTK](https://github.com/giubots/tini-mctk); the core idea remained the same: a developer can define a process and the framework guides the user through it regardless of the input type, supporting text and interface elements like buttons. This work was then presented at the [2022 International Conference on Advanced Visual Interfaces ](/publications/mctk-multi-modal).

Wanting to support more languages, I abandoned RASA in favour of [NLP.js](https://github.com/axa-group/nlp.js), which had also the advantage of running client-side in the browser. I also added multimodal output, so that not only the inputs can come from text or interface elements, but the interface can be altered according to the chatbot output. This work was presented at [Conversations 2022](/publications/enhancing-conversational-troubleshooting).

I forgot what TINI stands for officially, but in my mind it always meant "This Is Not an Issue", which is something a certain professor would always say when we were discussing another project I was working on, and at the same time it was a reference to the fact that this was a small component that could run in a browser.

## DiDisplay

The tagline was _"Effortless Real-Time Dialogue Display for Eye-Catching Demos"_; it started as a quick bug fix.
Working with the Pepper robot, a colleague of mine needed to display some text on the robot's tablet, but the text was not static: it was generated by a Python script running on a laptop, and it needed to be updated in real time.
On top of that, if the webpage was locally hosted, the robot would not display it, it needed a proper domain name and a valid SSL certificate.
In the first version, this app did just that: receive a string, display it (sanitisation nightmares aside).

It was then updated for [FTI 2023](/posts/three-tips-demo) with a better Vue frontend, and the ability to display full conversation history. It could also be configured with logos, and had some animations and exaggerated decorations to catch the public's attention. All packaged in a Docker container with health check endpoints.

![Screen recording of DiDisplay app, loading and displaying two messages.](https://giubots.net/projects/didisplay.gif)

## WebSocket Bridge

I needed to connect two websockets, neither of which had a fixed address.
[This express server](https://github.com/giubots/ws-bridge) acts as a bridge between the two, forwarding messages from one to the other and vice versa.
It can work for multiple connections: just connect one to `/ws/<foo>/<bar>` and the other to `/ws/<bar>/<foo>`, and they will be connected.
I packaged it in Docker with a nice health check, so it can be deployed easily.

## Devcontainer for PYNAOQI 2.5.5

Since I discovered devcontainers, I cant stop using them. When I needed to develop for NAO and Pepper robots, the setup was particularly annoying, so I created a [devcontainer](https://github.com/giubots/pynaoqi-devcontainer-2.5.5) for it.

## MrPenn

I needed an app to keep track of my expenses.
I had been using a sort of journal for some years, then I moved to a spreadsheet, but I wanted something faster to use especially on my phone.
I developed [MrPenn](https://github.com/giubots/mrpenn_flutter) between 2020 and 2023 in [Flutter](https://flutter.dev/).

The first functionality I implemented was a [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) for expenses; each with a title, amount, source and destination accounts, and a date.
Then, I added a view with the totals by account and category, a view with the list of expenses with filtering by text, and sorting by transaction or insertion date.
I was wise enough to then give priority to the SQLite database export, which saved me more than once. Always plan an exit strategy.

The feature I loved the most and the most useful was the ability to _flag_ expenses, and then view them in a separate list. I used this to keep track of expenses that I wanted to check later, like money lent or borrowed. Still haven't found a substitute for this feature in other apps.

I spent a bit of time on the UI, with a curved bottom navigation bar, and some animations.

![Screen recording of MrPenn app](https://giubots.net/projects/mrpenn.gif)

The name MrPenn is because at the time I was watching Gotham, a TV series withing the Batman universe, and the Penguin's accountant was called Arthur Penn.
I used MrPenn until 2025, when I finally started looking for an alternative with more features, and I switched to [Cashew](https://cashewapp.com/).

## TimedPool

I needed a dictionary with a maximum size and whose elements are deleted after a delay, so I made this [TimedPool package](https://github.com/giubots/timedpool).
It works like a normal dictionary, but if full, it raises an exception when trying to add a new element.
Elements can expire after a certain time, and expired elements are permanently removed from the dictionary.

## MultiModal Chatbot Creator (MMCC)

With two student colleagues, we worked on [MMCC](https://github.com/giubots/MultimodalChatbotCreator) for the Advanced User Interfaces and Multidisciplinary Project courses at Politecnico di Milano in 2021.
Starting from a preprint titled "A Conceptual Framework for Multi-modal Process-driven Conversational Agents", we implemented a framework that, if kept updated with the inputs from the user, guides him through an appropriately defined process, regardless of the input type.
The idea was that the developer could define a process, like a shopping experience, and use our components in their web page. The system would then provide a [RASA](https://rasa.com/)-powered backend that would handle the user input and guide them through the process, regardless of whether they were using text or interface elements like buttons, sliders, or images. The framework was designed to be easily extendable and adaptable to different use cases.

I focused on the core, implementing the logic that guides the user through a process, and calls appropriate callbacks when necessary. It could be used as a standalone library (with TKinter for example), or with the web components developed by my colleagues. This is an example of how the framework could be used in a web page:

```python
from mmcc_framework import Framework, RasaNlu

# Suppose that the nlu is trained with, among the others, the intent "insert_name" with a entity "name".
# Initialize the adapter
my_adapter = RasaNlu()
my_framework = Framework(..., nlu=my_adapter)

# Suppose that it is time to insert the name.

# If it is necessary to insert it as text use:
my_framework.handle_text_input("Mark")
# The callback corresponding to the current activity will receive (if the intent is recognized):
data = {"intent": "insert_name", "name": "Mark"}

# If it is necessary to insert the name as data use:
my_framework.handle_data_input(RasaNlu.dict("insert_name", {"name": "Mark"}))
# This will pass to the callback the same structure as above.
```

While working on MMCC, I taught myself Python best practices, notions about RASA NLU, and how to use Docker for development and deployment.
Looking back, what I handed in was very polished for a student project, with some testing, two examples and proper documentation.
I even published it as a Python package on [PyPI](https://pypi.org/project/mmcc-framework/).

## BarbequeRTRM Log Viewer

I developed [this](https://github.com/giubots/BarbequeRTRM-log-viewer) Qt plugin for [BarbequeRTRM](https://bosp.deib.polimi.it/) in 2020, for a university course at Politecnico di Milano.
The requirements were simple: open a log file, filter for text, log level, source module, and specific date ranges.
I added a button to quickly remove all filters, and everything was customisable, including date display format and log level colours.
After a few weeks of development, I presented the plugin to the professor and the supervisor of the course, and they seemed satisfied with the result. I was happy I got to learn about Qt and C++ in the process.

