Repository files navigation

Dicio is afree and open sourcevoice assistant running on Android. It supports many differentskills and input/output methods, and it provides bothspeech andgraphical feedback to a question. It interprets user input and (when possible) generates user output entirelyon-device, providing privacy by design. It hasmultilanguage support, and is currently available in these languages: Czech, Dutch, English, French, German, Greek, Italian, Polish, Russian, Slovenian, Spanish, Swedish and Ukrainian. Open to contributions :-D

Currently Dicio answers questions about:

• search: looks up information onDuckDuckGo (and in the future more engines) -Search for Dicio
• weather: collects weather information fromOpenWeatherMap -What's the weather like?
• lyrics: showsGenius lyrics for songs -What's the song that goes we will we will rock you?
• open: opens an app on your device -Open NewPipe
• calculator: evaluates basic calculations -What is four thousand and two times three minus a million divided by three hundred?
• telephone: view and call contacts -Call Tom
• timer: set, query and cancel timers -Set a timer for five minutes
• current time: query current time -What time is it?
• navigation: opens the navigation app at the requested position -Take me to New York, fifteenth avenue
• jokes: tells you a joke -Tell me a joke
• media: play, pause, previous, next song
• translation: translate from/to any language withLingva -How do I say Football in German?
• wake word control: turn on/off the wakeword -Stop listening

Dicio usesVosk as its speech to text (STT) engine. In order to be able to run on every phone small models are employed, weighing~50MB. The download fromhere starts automatically whenever needed, so the app language can be changed seamlessly.

Dicio's code isnot only here! The repository with thecompiler for sentences language files is atdicio-sentences-compiler, thenumber parser and formatter is atdicio-numbers and the code for evaluating matching algorithms is atdicio-evaluation.

When contributing keep in mind that other people may haveneeds andviews different than yours, so pleaserespect them. For any question feel free to contact the project team at@Stypox.

The#dicio channel onMatrix is available to get in touch with the developers:#dicio:matrix.org. Some convenient Matrix clients, available both for phone and desktop, are listed at that link.

If you want to translate Dicio to a new language you have to follow thesesteps:

• Translate thestrings used inside the app viaWeblate. If your language isn't already there, add it withtool -> start new translation.
  

• Translate thesentences used by Dicio to identify a user's request and to feed it to the correct skill. To do this open the repository root and navigate toapp/src/main/sentences/. Copy-paste theen folder (i.e. the one containing English translations) and call the new folder with the 2- or 3-letter name of your language (in particular, anyISO-639-compliant language ID is supported). Then open the newly created folder: inside there will be some YAML files in English language. Open each one of them and translate the English content; feel free to add/remove sentences if their translation does not fit into your language and remember those sentences need to identify as better as possible what the user said. DoNOT change the name of the copied files, the IDs of the sentences (i.e. thesentence_id: before each list of sentences) or the IDs of the capturing groups (i.e. the.ID. construct). To learn about the Dicio sentences language syntax, please refer to the documentation and theexample indicio-sentences-compiler. Hopefully in the future a custom translation system will be used for sentences.

• Once both the Weblate and the sentences translations are ready, add the new language to the app's language selector. This requires modifying two files. In the following instructions$iso-language-id$,$ISO_LANGUAGE_ID$ and$Language name$ are placeholders you should fill in, e.g.en-in,EN_IN andEnglish (India).

  1. inapp/src/main/proto/language.proto add a new entry to theLanguage enum like so:LANGUAGE_$ISO_LANGUAGE_ID$ = $increasing number$; // $Language name$. Use the previous highest number in the enum incremented by 1 as$increasing number$. The position of the newly added item should make it so that the items are sorted by their language name.
  2. inapp/src/main/kotlin/org/stypox/dicio/settings/Definitions.kt add a new item tolanguageSetting.possibleValues like so:ListSetting.Value(Language.LANGUAGE_$ISO_LANGUAGE_ID$, "$Language name$"),. The position of the newly added item should make it so that the items are sorted by their language name.

• Then update the app descriptions so that people know that the language you are adding is supported. The files you should edit areREADME.md (i.e. the file you are currently viewing) andfastlane/metadata/android/en-US/full_description.txt (the English description for F-Droid).

• Open a pull request containing both the translated sentences files, the language selector addition and the app descriptions updates. You may want to take a look atthe pull request that added German, #19, and if you need help don't hesitate to ask :-)

A skill is a component that enables the assistant tounderstand some specific queries andact accordingly. While reading the instructions, keep in mind thejavadocs of the methods being implemented and the code of the already implemented skills. In order to add a skill to Dicio you have to follow the steps below.

_{$skill_id$ and$SkillId$ indicate the computer readable name of the skill, in snake_case and PascalCase, e.g.weather orWeather}

The new skill most likely needs to interpret user input. The Dicio framework provides astandard way to define how to efficiently match user input and extract information from it, in the form of translatable reference sentences stored in YAML files. Note thatfor some specific cases the standard recognizer might not be wanted, in which case you can skip this section, and in section 3 extendSkill<> and implementSkill.score() manually, instead of extendingStandardRecognizerSkill<>.

1. Edit theapp/src/main/sentences/skill_definitions.yml file and add a definition for the new skill:

   # The unique ID of the skill.-id:$skill_id$# `SPECIFICITY` can be `high`, `medium` or `low`.# It should be chosen wisely: for example, a section that matches queries# about phone calls is very specific, while one that matches every question# about famous people has a lower specificity.specificity:SPECIFICITY# A list of definitions for the types of sentences this skill can interpret.# Can contain multiple sentences, e.g. the timer skill has the# "set", "cancel" and "query" sentences.sentences:# An ID for the sentence, must be unique amongst this skill's sentences.    -id:SENTENCE_1_ID# (optional) If this sentence has some capturing groups, their IDs and# types must be listed here.captures:# An ID for the capturing group, must be unique amongst this# sentence's capturing groups        -id:CAPTURING_GROUP_1_ID# Currently only string capturing groups are supported, but in# the future "number", "duration" and "date" will also be possible.# For the moment use "string" and then manually parse the string to# number, duration or date using dicio-numbers.type:string

2. Create a file named$skill_id$.yml (e.g.weather.yml) underapp/src/main/sentences/en/: it will contain thesentences the skill should recognize.

3. For each of the sentence definitions inskill_definitions.yml, write the id of each sentence type followed by: and a list of sentences:

   SENTENCE_1_ID:  -a<n?> sentence|phrase? alternative# ...  -another sentence|phrase? alternative with .CAPTURING_GROUP_1_ID.# ...# ...# SENTENCE_2_ID: ... in case you have multiple sentence types

4. Write the reference sentences according to thedicio-sentences-language's syntax.

5. Try tobuild the app: if it succeeds you did everything right, otherwise you will get errors pointing to syntax errors in the.yml files.

Here is an example of the weather skill definition inskill_definitions.yml:

-id:weatherspecificity:highsentences:    -id:currentcaptures:        -id:wheretype:string

And these are the example contents ofapp/src/main/sentences/en/weather.yml:

current:  -(what is|s)|whats the weather like? (in|on .where.)?  -weather (in|on? .where.)?  -how is it outside

Create asubpackage that will contain all of the classes you are about to add:org.stypox.dicio.skills.SKILLID (e.g.org.stypox.dicio.skills.weather).

Create a class named$SkillId$Skill (e.g.WeatherSkill): it will contain the code that interprets user input (i.e. thescore() function) and that processes it to generate output (i.e. thegenerateOutput() function). The next few points assume that you want to use thestandard recognizer with the skill definition and sentences you created instep 1. In that casescore() is actually already implemented and you don't need to provide an implementation yourself.

1. Have the$SkillId$Skill class implementStandardRecognizerSkill<$SkillId$>. You can import the$SkillId$ class withimport org.stypox.dicio.sentences.Sentences.$SkillId$. TheSentences.$SkillId$ sealed class is generated based onskill_definitions.yml, and contains one subclass for each of the defined sentence types.
2. The constructor ofSkill takesSkillInfo (seestep 5), and moreover the constructor ofStandardRecognizerSkill takesStandardRecognizerData<$SkillId$> (the data generated from the sentences, seestep 5). You should expose these two parameters in$SkillId$Skill's constructor, too.
3. Implement the following function:override suspend fun generateOutput(ctx: SkillContext, inputData: $SkillId$): SkillOutput.inputData is, again, an instance ofSentences.$SkillId$ corresponding to the matched sentence type, and its fields contain type-safe information about the data captured in capturing groups (if any).
4. Any code makingnetwork requests or heavy calculations should be put ingenerateOutput (which is a suspend function for this exact purpose). The returnedSkillOutput should contain all of the data needed to actually show the output, i.e. it shouldn't do any more network requests or calculations (unless it's an interactive widget and the user presses some button, but that's not too relevant for the matter at hand).

This is a stub implementation of theWeatherSkill:

packageorg.stypox.dicio.skills.weatherimportorg.stypox.dicio.sentences.Sentences.Weather// ...classWeatherSkill(correspondingSkillInfo:SkillInfo,data:StandardRecognizerData<Weather>) :StandardRecognizerSkill<Weather>(correspondingSkillInfo, data) {overridesuspendfungenerateOutput(ctx:SkillContext,inputData:Weather):SkillOutput {return// ...    }}

Create a class named$SkillId$Output (e.g.WeatherOutput): it will contain the code that creates a Jetpack Compose UI and provides speech output.

1. The class should be constructed bySkill.generateOutput() with all of the data needed to display/speak output, and is meant to be serializable (so in most cases it is adata class). In some cases it might make sense to have multiple types of output (e.g. the weather hasSuccess andFailed output types): in that case you can create asealed interface and have both output types extend it.
2. getSpeechOutput() returns a localized string that will be spoken via the configured Text To Speech service.
3. @Composable GraphicalOutput() builds the UI that will be shown in a box on the home screen. The UI can be interactive and can act as a widget: for example the timer skill shows the ongoing countdown.
4. [Optional]getNextSkills() returns a list of skills that could continue the current conversation. If this list is non-empty, the next time the user asks something to the assistant, these skills will be considered before all other skills, and if any of these skills understands the user input well enough, the conversation continues. For example, if the user says "Call Mom", the assistant may answer with "Should I call mom?" and this method would return a skill that can understand a yes/no response.

This is a stub implementation ofWeatherOutput:

data classWeatherOutput(valcity:String,valdescription:String,// ...) : WeatherOutput {overridefungetSpeechOutput(ctx:SkillContext):String= ctx.getString(R.string.skill_weather_in_city_there_is_description, city, description    )    @ComposableoverridefunGraphicalOutput(ctx:SkillContext) {// Jetpack Compose UI    }}

Create anobject named$SkillId$Info (e.g.WeatherInfo) overridingSkillInfo: it will contain all of theinformation needed to manage your skill.

1. This is not a class, but anobject, because it makes no sense to instantiate it multiple times.
2. Call theSkillInfo constructor with the"$skill_id$" string.
3. Provide sensible values forname(),sentenceExample() andicon().
4. Override theisAvailable() method and return whether the skill can be used under thecircumstances the user is in (e.g. check whether the recognizer sentences are translated into the user language withSentences.$SkillId$[ctx.sentencesLanguage] != null (seestep 1 and step 5.5) or check whetherctx.parserFormatter != null, if your skill uses number parsing and formatting).
5. Override thebuild() method so that it returns an instance of$SkillId$Skill with$SkillId$Info ascorrespondingSkillInfo and, if the skill uses standard recognizer sentences (seestep 1), withSentences.$SkillId$[ctx.sentencesLanguage] asdata.
6. [Optional] If your skill wants to present some preferences to the user, it has to do so by overriding therenderSettings value (which by default returnsnull to indicate there are no preferences).

Underorg.stypox.dicio.Skills.SkillHandler, update theallSkillInfoList by adding$SkillId$Info; this will make the new skill finally visible to Dicio.

• Thectx: SkillContext object, that appears here and there in the implementation, allows accessing the Android context, the number parser/formatter and otherresources and services, similarly to Android'scontext.
• Thenames used for things (files, classes, packages, sections, etc.) are not mandatory, but they helpavoiding confusion, so try to stick to them.
• When committing changes about a skill, prefix the commit message with "[$SkillId$]", e.g. "[Weather] Fix crash".
• Add your skill with a short description and an example in the README underSkills and in thefastlane's long description.
• If you have any question,don't hesitate to ask. 😃