Custom types with payload
Elm makes it easier to describe complex data structures by letting us add a payload to each value in a custom type. To understand what a payload is, let’s add a couple more options to our Greeting type.
We added two more ways to create a value of type Greeting. Namaste enables us to say hi in Nepali language, and NumericalHi allows us to greet mathematicians.
Unlike Howdy and Hola, Namaste and NumericalHi aren’t values by themselves.
Interestingly enough, data constructors are actually functions behind the scenes. They take a payload as an argument and create values of type Greeting. In case of Namaste, the payload only consists of a string, but the payload for NumericalHi includes two Int values. There’s no limit to how many and what type of data a payload can contain.
Namaste and NumericalHi are like functions in that they must be applied to the arguments inside their payloads to create concrete values.
Note: In Nepali language, “Tapailai kasto cha?” means “How are you?”.
It’s important to keep in mind that the data constructors don’t behave like normal functions in terms of performing an operation on data. They’re more like boxes to put data into. They don’t do anything with that data other than carry them around. That’s why when we type NumericalHi 1 4 into the repl, it spits out the same thing back.
Since Howdy and Hola take no payload, their values don’t need to be constructed. Their values are already established, which essentially makes them constants. That’s why when we print them, the repl doesn’t present them to us as functions.
The way we used Namaste and NumericalHi in the sayHello function is very similar to how we used Howdy and Hola.
Although Howdy and Hola behave like constants, sometimes you’ll hear them being referred to as nullary data constructors.
Example
For instance, a plugin named plugins/task_routine.py defines the task task_routine, which is configured in the dictionary element ‘A_ROUTINE’, like the following example:
'UDS_START_DIAG_SESS': {
'Request': '^1085' ELM_FOOTER, # 85 = Flash Programming Session'Descr': 'UDS Start Diagnostic Session',
'Response': PA('') # Response: 50 85
},
'UDS_REQ_SEED': { # Start seed & key and request the seed from ECU'Request': '^2701' ELM_FOOTER,
'Descr': 'UDS SecurityAccess - requestSeed',
'Response': PA('12 34') # Response: 67 01 12 44; seed is 12 44
},
'UDS_SEND_KEY': {
'Request': '^2702' ELM_DATA_FOOTER,
'Descr': 'UDS SecurityAccess - Send Key to ECU',
'Exec': 'self.shared.auth_successful = cmd[4:] == "3322"', # Key'Info': '"auth_successful: %s", self.shared.auth_successful',
'ResponseFooter': lambdaself, cmd, pid, uc_val: (
PA('') ifself.shared.auth_successfulelseNA('35')
) # Response: 67 02 if pos.; 7F 27 35 if neg. 35=invalidKey
},
'A_ROUTINE': { # UDS Routine Control (31): Start (01)'Request': '^3101' ELM_DATA_FOOTER,
'Descr': 'An UDS Routine',
'Task': 'task_routine'
},
The following table shows the UDS protocol sequence for this example:
Application (request) | ECU (response) | Protocol | Description |
---|---|---|---|
10 85 | 50 85 | StartDiagnosticSession, ECUProgrammingMode | Start Programming Mode (ECU returns a positive answer) |
27 01 | 67 01 12 44 | SecurityAccess, requestSeed | Start seed & key and request seed to ECU (ECU returns a positive answer, the seed 12 44 is then sent back from the ECU to the application) |
27 02 33 22 | 67 02 | SecurityAccess, sendKey | Send security key 33 22 to ECU (ECU returns a positive answer) |
31 01 | 71 01 | StartRoutineByLocalIdentifier, ID = 01 (Start) | Start flash driver download into RAM (ECU returns a positive answer) |
In such example, a request of type 3101… will start the task task_routine, which can immediately return (if Tasks.RETURN.TERMINATE is used), or (in case Tasks.RETURN.CONTINUE is used) will also be able (not in this example of task) to process any subsequent request (also if not matching 3101…), until the plugin is terminated. In the above example, the ‘Header’ attribute is not set, so any CAN header will be valid.
Function type
Functions also have types.
We defined a function called addOne that takes one argument of type number and returns a number as well. So its type is printed as number -> number.
When we enter a function definition into a code file, it’s best practice to write down its type annotation. Add the following function definition right above main in Playground.elm.
The parameter and return types are separated by ->. We didn’t specify the type annotation for any of the functions we created before. How was Elm able to correctly identify the types of parameters and return values without the type annotation?
Elm was able to do that because it can infer the types based on what operations we perform inside a function. In addOne, we’re using the operator which has the following type.
It takes two numbers and returns a number. The parameter x in addOne must be a number to satisfy the operator’s constraints.
This automatic deduction of types is known as type inference. Elm makes extensive use of type inference throughout our code base so that we don’t have to specify the type of each and every value used in our programs. Let’s look at a few more examples of type inference in the repl.
When we use the floating-point division operator (/), the divideByTwo function’s type is inferred as Float ->
Float, but when we use the integer division operator (//), which truncates everything after the decimal point, the type is inferred as Int -> Int.
So far we’ve looked at simple functions that only use one operator. Let’s write a slightly more complex function and find out if Elm can infer its type too. Add the following function definition right above main in Playground.elm.
And expose guardiansWithShortNames in the module definition.
We didn’t specify the type annotation for guardiansWithShortNames because we want Elm to infer its type. Let’s see what Elm shows as its type in the repl.
The first operation we apply to the guardians parameter helps Elm determine its type. For List.map to be able to apply String.length to each element in a list, those elements must be of type String.
That’s why we see List String as the guardian parameter’s type. Similarly, Elm determines the return type of a function from the last operation performed.
Therefore, Elm deduces that the guardiansWithShortNames function also returns an Int value. If Elm infers type automatically, then why do we still need to write the type annotation above each function declaration? Because we get the following benefits from type annotations:
- Provides documentation
- Enables code validation
- Limits types of values a function can accept
Multiple type arguments
Elm allows us to have multiple arguments in a type definition. Here’s an example:
Like Maybe, Result is another built-in type in Elm. It accepts two type arguments: error and value.
The Result type comes very handy when an operation fails and we need to return a descriptive error message. To see it in action, let’s try to decode some JSON values and see what output we get.
If you see the following message, answer Y.
Now run the following code in elm repl.
Note: The Json.Decode module provides functions for decoding a JSON string into Elm values. We’ll cover this module in detail in chapter 6.
When we give the decodeString function an invalid input, it returns a value of type Result instead of crashing our program. Here’s what decodeString’s type signature looks like:
To create a value of type Result, we must use one of these data constructors: Ok and Err.
Result type is a bit more expressive than Maybe. Instead of just returning Nothing, we can create a descriptive message that explains why the operation didn’t succeed.
The next example shows how the Result type can make the output of our function more descriptive. Add the following function definition right above main in Playground.elm.
Now expose signUp in the module definition.
Obdmessage dictionary generator for “elm327-emulator” (obd_dictionary)
obd_dictionary is a dictionary generator for “ELM327-emulator”.
It queries the vehicle via python-OBD for all available commands and is also able to process custom PIDs described in Torque CSV files.
Its output is a Python ObdMessage dictionary that can either replace the obd_message.py module of ELM327-emulator, or extend the existing dictionary via merge command, so that the emulator will be able to provide the same commands returned by the vehicle.
Notice that querying the vehicle might be invasive and some commands can change the car configuration (enabling or disabling belts alarm, enabling or disabling reverse beeps, clearing diagnostic codes, controlling fans, etc.). In order to prevent dangerous PIDs to be used for building the dictionary, a PID blacklist (blacklisted_pids) can be edited in elm.py.
obd_dictionary can be run as:
python3 -m obd_dictionary --help
or simply:
Command line arguments:
usage: obd_dictionary [-h] -i DEVICE [-c CSV_FILE] [-o FILE] [-v] [-V] [-p PROBES] [-B BAUDRATE]
[-T TIMEOUT] [-C] [-F] [-P PROTOCOL] [-d DELAY] [-D DELAY_COMMANDS] [-n CAR_NAME]
[-b] [-r] [-x] [-t [FILE]] [-m]
optional arguments:
-h, --help show this help message and exit
-i DEVICE python-OBD interface: serial port connected to the ELM327 adapter (required
argument).
-c CSV_FILE, --csv CSV_FILE
input csv file including custom PIDs (Torque CSV Format: https://torque-
bhp.com/wiki/PIDs) '-' reads data from the standard input
-o FILE, --out FILE output dictionary file generated after processing input data (replaced if
existing). Default is to print data to the standard output
-v, --verbosity print process information
-V, --verbosity_debug
print debug information
-p PROBES, --probes PROBES
number of probes (each probe includes querying all PIDs to the OBD-II adapter)
-B BAUDRATE, --baudrate BAUDRATE
python-OBD interface: baudrate at which to set the serial connection.
-T TIMEOUT, --timeout TIMEOUT
python-OBD interface: specifies the connection timeout in seconds.
-C, --no_check_voltage
python-OBD interface: skip detection of the car supply voltage.
-F, --fast python-OBD interface: allows command optimization (CR to repeat, response limit).
-P PROTOCOL, --protocol PROTOCOL
python-OBD interface: forces using the given protocol when communicating with the
adapter.
-d DELAY, --delay DELAY
delay (in seconds) between probes
-D DELAY_COMMANDS, --delay_commands DELAY_COMMANDS
delay (in seconds) between each PID query within all probes
-n CAR_NAME, --name CAR_NAME
name of the car (dictionary label; default is "car")
-b, --blacklist include blacklisted PIDs within probes
-r, --dry-run test the python-OBD interface in debug mode.
-x, --noautopid do not autopopulate the pid list with the set of built-in commands supported by
the vehicle; only use csv file.
-t [FILE], --at [FILE]
include AT Commands within probes. If a dictionary file is given, also extract AT
Commands from the input file and add them to the output
-m, --missing add in-line comment to dictionary for PIDs with missing response
ObdMessage Dictionary Generator for "ELM327-emulator".
Sample usage: obd_dictionary -i /dev/ttyUSB0 -c car.csv -o AurisOutput.py -v -p 10 -d 1 -n mycar
obd_dictionary exploits the command discovery feature of python-OBD, which autopopulates the set of builtin commands supported by the vehicle through queries performed within the connection phase.
The command allows all the python-OBD interface settings (see -B, -T, -C, -F, -P command-line options) and a dry-run flag (-r), which is very useful to test the OBD-II connection.
For instance, the following command tests an OBD-II connection via Bluetooth using the related recommendations described in the python-OBD repository.
python3 -m obd_dictionary -i /dev/rfcomm0 -B 38400 -T 30 -r
See also this post for Bluetooth.
For better analysis, the -r output can be piped to lnav (the following command tests the USB connection):
When the tests provide successful connection, the -r option can be removed and the additional obd_dictionary options can be added.
obd_dictionary can be also used to test elm. Run python3 -m elm -s car. Read the pseudo-tty, say /dev/pts/2 (this mode uses the serial communication). Run obd_dictionary:
(The automation of this process is shown further on.)
In general, ELM327-emulator should already manage all needed AT Commands within its default dictionary, so in most cases it is worthwhile removing them from the new scenario via -t option.
The file produced by obd_dictionary provides the same information model of obd_message.py. It can be used to replace the default module or can be dynamically imported in ELM327-emulator through the merge command, which loads an ObdMessage dictionary and merges it to emulator.ObdMessage. Example of merge process:
To help to configure the emulator, autocompletion is allowed (by pressing TAB) when prompting the merge command, including the merge argument. Also variables and keywords like scenario accept autocompletion, including the scenario argument.
A merged scenario can be removed via del emulator.ObdMessage[‘<name of the scenario to be removed>’].
To produce a complete dictionary file that can replace obd_dictionary:
Recursive types
In the How List Works Behind the Scenes section, we learned that List in Elm is defined as a recursive type. At the time, we didn’t know enough about types to fully grasp the idea of recursive types.
Now that we know what types are, we’re better positioned to understand what they are. Let’s say we have a list of numbers: [ 16, 5, 31, 9 ]. Behind the scenes, this list is constructed like this:
We started with an empty list and added 9 in-front of it using the cons (::) operator. We then continued to add the rest of the elements to that list one at a time. When a list is constructed like this, we can see the underlying recursive structure inherent in all lists.
The figure above shows that a list consists of nodes which themselves are lists. This is what makes the List type recursive. Let’s create our own data structure that looks very much like List to better understand how a recursive data structure works.
And expose MyList in the module definition.
What the definition above means is that a list of type MyList can be either Empty or Node a followed by another list (MyList a).
A list with no elements can be represented like this: Empty. A list with a single element is represented like this: Node a Empty. Similarly, a list with two elements is represented like this: (Node a (Node a Empty)) and so on.
We start with an empty element and then add 9 in front of it similarly to how we built a list using the cons (::) operator: 9 :: []. Next, we keep adding the rest of the elements to the front.
Granted our list doesn’t look as nice as the one Elm provides, but conceptually they’re the same. Although MyList behaves similarly to List, we can’t use any of the functions defined in the List module. MyList and List are two completely different types.
In the Easier Code Organization section we will reimplement one of the functions from the List module so that it will work on MyList too.
It’s important to note that if a recursive type doesn’t provide at least one nullary data constructor, then we end up with a value that never ends. If we remove the Empty data constructor from MyList:
We’ll end up with a value like this:
… represents infinite iterations of Node a.
Type annotation with multiple parameters
The type annotation for a function that accepts multiple arguments can be confusing to look at.
The return type is separated from the parameters by ->. The parameters themselves are also separated by ->. There’s no visual cue to tell where the parameters end and the return type begins.
In the Partial Function Application section, we learned that when we don’t pass enough arguments to a normal function, instead of getting an error we get a partially applied function.
When we pass only the first argument to add, it returns a function that looks something like this behind the scenes:
It replaced num1 with 1 and now it’s waiting for us to pass the second argument. First, let’s assign the partially applied function to a constant.
Now we can apply addPartiallyApplied to 2 and get our final result.
In the beginning, add looked like a function that took two arguments and returned an Int value, but after careful inspection we found out that it actually accepts only one argument and returns a partially applied function.
All functions in Elm work in this manner no matter how many arguments they appear to take on the surface. With this new found knowledge, we can rewrite the add function’s type annotation like this:
Since parentheses indicate a function, if we continue to wrap each individual function in parenthesis, the type annotation will look like this:
However, Elm makes all those parentheses optional so that our type annotations can look much cleaner. That’s how we ended up with this:
- Currying
- This process of evaluating a function that takes multiple arguments by converting it to a sequence of functions each taking a single argument is known as currying. This technique is what enables us to use the
|>
operator. Here’s an example we saw back in the Function section:
Type constructor
In the Regular Expressions section we learned how the Maybe type works, but we haven’t really seen its type definition. Here is how it looks:
Maybe is a built-in type in Elm that allows us to express the idea of a missing value. Often times we are not quite sure whether a value we are looking for really exists. For example, if we try to retrieve the tenth element from an array that only contains five elements, we get Nothing.
Instead of returning an error or crashing our program, the get function returns a value of type Maybe. Here is what the type annotation for get looks like:
Like List and Array, Maybe is a container, but it can have at most one value in it.
That value can be of any type. To create a value of type Maybe, we must use either the Just data constructor or Nothing constant.
Unlike our Greeting type, Maybe by itself is not a valid type. It merely provides a way for us to construct a type. That’s why it’s called a type constructor.
It must be applied to another type argument for it to generate a valid type. Maybe Int, Maybe String, Maybe (List number) are all valid types.
Generic (or parameterized) types such as Maybe a can be incredibly powerful. To create our own generic types all we have to do is pass an argument to a type constructor. The Greeting type we created earlier is not a generic type.
Data constructors that create a value of type Greeting expect their payloads to be of certain types. Namaste requires its payload to be a String, and NumericalHi requires two Int values.
Now that we can pass a type argument to Greeting, the Namaste data constructor isn’t limited to just one type. It accepts a payload of any type.
Before trying out the following examples, comment out the sayHello function including its type annotation and remove it from the list of values being exposed in the module definition. Otherwise, you’ll get errors. We’ll fix that function soon.
Its type signature has also changed.
Before adding a type argument, it was this:
Notice that we used the type argument a only with Namaste, but not with NumericalHi.
We’re not required to pass a type argument to all data constructors. In fact, we don’t even have to pass it to any data constructor. The following type definition is perfectly valid in Elm. Modify the type definition for Greeting to look like this:
A type argument that doesn’t get used in any of the data constructors is known as a phantom type argument. There are legitimate reasons for a phantom type argument’s existence, but the explanation of those reasons is beyond the scope of this book.
Since Greeting and Greeting a are two different types, we need to modify the sayHello function’s type annotation.
We only need to change the type annotation, but not sayHello greeting = part in function definition, because the function parameter greeting simply holds any value of type Greeting a.
Another way of looking at it is, if we change our type definition to type Welcome a, we would need to change the type annotation to sayHello :
The type annotation is the thing that connects the value stored in a function parameter to the correct type, not the name of the function parameter itself, which can be whatever we want. Before we move on, let’s revert the type definition for Greeting back to this:
The only thing that changed is the Namaste data constructor requires its payload to be of type a instead of String.
According to the error message, all branches in a case expression must return a value of the same type, but we aren’t following that rule. We’re returning a String value for Howdy, Hola, and NumericalHi, and returning a value of any type (represented by a) for Namaste.
Functions in Elm must return a value of only one type. Therefore, we need to either have all branches return a value of type a or String. Let’s revert the definition back to what it was before we introduced the type variable to get rid of the error:
Since Greeting doesn’t accept a type variable anymore, we need to modify the sayHello function’s type annotation to this:
Type vs data constructor
At this point you may be wondering where exactly in our code do type and data constructors get used. Type constructors are mainly used either in a type declaration or a type annotation, whereas data constructors are used inside a function body or when we define a top-level constant.
Let’s say we want to find out which of the Stark siblings from Game of Thrones have reached adulthood. Add the following code right above main in Playground.elm.
And expose Character, sansa, arya, and getAdultAge in the module definition.
We defined a record called Character that contains a character’s name and age. A concrete type Maybe Int is assigned to the age field to indicate that a character may choose not to disclose their age.
We then created two characters: sansa and arya. sansa’s age is included in the record as Just 19, but arya’s is listed as Nothing which means her age is unknown.
So far, in the example above, we have encountered one type constructor (Maybe) and two data constructors (Just and Nothing). Nothing is more like a constant, but we are treating it as a nullary data constructor here.
Since Character is a type alias, it’s not considered a real type. As we can see, the Maybe type constructor is used in the record’s type declaration.
Although Character is not a real type, { name : String, age : Maybe Int } is. A type alias tends to show up in places where a type constructor usually does.
When we create an actual record, instead of using the Maybe type constructor, we need to use either Just or Nothing data constructor.
The Maybe type constructor also shows up in the getAdultAge function’s type annotation.
The Just and Nothing data constructors are used inside the function body to create actual values of type Maybe Int.
Let’s see how the getAdultAge function behaves when we give it a character whose age is present.
In repl, a data constructor shows up in the value area, whereas a type constructor shows up in the type annotation area.
How about a character whose age is missing?
As expected, we get Nothing. Let’s create three more characters in repl to further explore the getAdultAge function’s behavior.
What happens if we give a character whose age is less than 18 to getAdultAge?
Instead of returning rickon’s actual age, it returns Nothing because the getAdultAge function is designed to ignore the age of characters who haven’t reached adulthood yet. We can take advantage of this behavior to do something cool: print only the age of adult characters.
In Elm, we can guess how a function works by looking at its type. Let’s give it a try. When we ask the repl for List.filterMap’s type, here’s what we get:
The type annotation tells us that the filterMap function takes two arguments:
Finally, the filterMap function returns a list of values of type b. In most cases, type annotation alone isn’t enough to figure out how a function actually works.
For example, one of the behaviors that’s not quite apparent from the type annotation is that filterMap discards all elements from the original list for which the getAdultAge function returns Nothing.
Using custom types
Let’s see how we can use the new type Greeting in our code. Add the following code right above the main function in Playground.elm located in the beginning-elm/src directory.
And expose Greeting(..) and sayHello in the module definition.
Note: To access Howdy and Hola from outside the Playground module, we have to add (..) after Greeting in the module definition. (..) tells Elm to expose all data constructors inside a type. More on this later.
A custom type is often used with a case expression to pattern match a value of that type. Once a match is found, the corresponding expression is evaluated. There’s nothing special about how we use a custom type. The Bool type provided by Elm can also be used in a similar fashion.
The sayHello function takes one argument of type Greeting. If the value is Howdy, it engages in a proper Texan interaction.
If you see the following error, restart the elm repl session. We defined the Greeting type first in the repl. Later when we redefined it in Playground.elm, the repl gets confused — hence the error message.
When typing code in a file, it’s best practice to break the definition of a custom type so that each value gets a line of its own. That’s why when we enter type Greeting = Howdy | Hola into an Elm file and save, elm-format automatically reformats it to:
In contrast, when typing code into the repl, it’s not necessary to break it into multiple lines because typing multiline code in the repl is tedious and requires special characters to indicate a line break.
Working with recursive types
We can use recursive types the same way we use any other union type. A case expression is used to pattern match each individual data constructor defined in the type. Add the following function definition right above main in Playground.elm
And expose sum in the module definition.
sum is a function that computes a sum of int values contained in a MyList. We need to handle only two cases (Empty and Node) because those are the only two data constructors MyList type offers.
If the list is not empty, we remove an int value from the front and apply sumrecursively to the rest of the list. Here’s how we can use sum in the repl:
The figure below shows each individual step in the execution of sum myList expression.
Recursive types are very powerful. They enable us to define complex data structures succinctly. Let’s implement one more data structure called binary tree using a recursive type. Explaining the inner workings of a binary tree is out of scope for this book, so we’ll just look at its definition and visual illustration.
Tree represents a binary tree — a hierarchical tree structure in which each node can have at most two children. It has many applications in programming.
The tree shown in the figure above can be implemented like this: