Working with J1939 Data

Python 3 tools and code snippets to transorm, parse, decode, and simulate heavy vehicle network data on J1939 Networks. The CAN Data page has recordings of heavy vehicle data.

Overview

The idea of J1939 is to build a reliable communication on top of a Controller Area Network (CAN) using 29-bit arbitration identifiers. Working with the J1939 network requires performing some of the following tasks:

For the practical work described in this page, we need to have some hardware. Since the examples are in Python 3, Linux SocketCAN, and can-utils, we will use hardware that runs an embedded Linux operating system. The BeagleBone Black with a TruckCape is our prefered device. A Raspberry Pi with a CAN hat would also work.

Heavy Truck Cape Image

Reading J1939 Traffic

Once you have a some hardware in hand and have access to a truck or J1939 network, you'll need to follow these steps:

  1. Login to a terminal on the Linux device.
    For example, follow the instructions for the truck cape.
  2. Confirm the can interfaces are present:
    ifconfig
  3. Confirm the bitrate for CAN
    ip -details link show can1
    where can1 is the physical channel connected to J1939. Check all valid CAN channels.
  4. If the bitrate needs to be changed, use the following 3 commands:
    sudo ip link set can1 down
    sudo ip link set can1 type can bitrate 250000
    sudo ip link set can1 up
    Note: you can omit the middle command to simply reset the interface.
  5. Use the candump command to capture network traffic without processing it.
    candump -s0 -l any
    where the -l switch creates a log file and -s0 turns off silent mode so you can see what's going on. when using 'any' for the interface means the command will log traffic from all channels. Just typing candump with no arguments gives the help documentation. Note: the format displayed on the screen is different than the candump file format.

What if we wanted to not only read, but also process the data in real time? This would be important to respond to requests or commands that are coming from the CAN bus. To do this, we'll need to write a program to process and read the data.

Let's start by writing a simple Python script to write a candump log file. An example of this is in the TruckCapeProjects repository.

#!/usr/bin/python3
import socket
import struct
import time

# Open a socket and bind to it from SocketCAN
sock = socket.socket(socket.PF_CAN, socket.SOCK_RAW, socket.CAN_RAW)
interface = "can1"

# Bind to the interface
sock.bind((interface,))

# To match this data structure, the following struct format can be used:
can_frame_format = "<LB3x8s"

with open("candump_python.log", 'w') as outfile:
    while True:
        can_packet = sock.recv(16)
        can_id, can_dlc, can_data = struct.unpack(can_frame_format, can_packet)
        extended_frame = bool(can_id & socket.CAN_EFF_FLAG)
        if extended_frame:
            can_id &= socket.CAN_EFF_MASK
            can_id_string = "{:08X}".format(can_id)
        else: #Standard Frame
            can_id &= socket.CAN_SFF_MASK
            can_id_string = "{:03X}".format(can_id)
        hex_data_string = ' '.join(["{:02X}".format(b) for b in can_data[:can_dlc]])
        print("{} {} [{}] {}".format(interface, can_id_string, can_dlc, hex_data_string))
        outfile.write("({:12.6f}) {} {}#{}\n".format(time.time(),interface, can_id_string, hex_data_string.replace(" ","")))
    

Let's assume we name the above script as 'candump_example.py'. Then it can be run at the command prompt as python3 candump_example.py. The output to both the screen and the file will match the candump format from can-utils. Since it is in a while True loop, press ctrl-C to stop.

This simple script only works for one interface. If we want more interfaces, then it makes sense to convert this to a class an instantiate a python object for each interface that's available. This class can be threaded, and part of a bigger program to work with the CAN data.

Decoding the CAN ID for J1939

A very useful tool to decode the CAN message is the pretty_j1939 project.

We take advantage of this database in json form when we work through the examples in the TruckCape Repository. In fact, this repository has all the solutions needed to work with the heavy vehicle CAN systems.