1 .. SPDX-License-Identifier: GPL-2.0-or-later
6 Author: Martin Jerabek <martin.jerabek01@gmail.com>
9 About CTU CAN FD IP Core
10 ------------------------
12 `CTU CAN FD <https://gitlab.fel.cvut.cz/canbus/ctucanfd_ip_core>`_
13 is an open source soft core written in VHDL.
14 It originated in 2015 as Ondrej Ille's project
15 at the `Department of Measurement <https://meas.fel.cvut.cz/>`_
16 of `FEE <http://www.fel.cvut.cz/en/>`_ at `CTU <https://www.cvut.cz/en>`_.
18 The SocketCAN driver for Xilinx Zynq SoC based MicroZed board
19 `Vivado integration <https://gitlab.fel.cvut.cz/canbus/zynq/zynq-can-sja1000-top>`_
20 and Intel Cyclone V 5CSEMA4U23C6 based DE0-Nano-SoC Terasic board
21 `QSys integration <https://gitlab.fel.cvut.cz/canbus/intel-soc-ctucanfd>`_
22 has been developed as well as support for
23 `PCIe integration <https://gitlab.fel.cvut.cz/canbus/pcie-ctucanfd>`_ of the core.
25 In the case of Zynq, the core is connected via the APB system bus, which does
26 not have enumeration support, and the device must be specified in Device Tree.
27 This kind of devices is called platform device in the kernel and is
28 handled by a platform device driver.
30 The basic functional model of the CTU CAN FD peripheral has been
31 accepted into QEMU mainline. See QEMU `CAN emulation support <https://www.qemu.org/docs/master/system/devices/can.html>`_
32 for CAN FD buses, host connection and CTU CAN FD core emulation. The development
33 version of emulation support can be cloned from ctu-canfd branch of QEMU local
34 development `repository <https://gitlab.fel.cvut.cz/canbus/qemu-canbus>`_.
40 SocketCAN is a standard common interface for CAN devices in the Linux
41 kernel. As the name suggests, the bus is accessed via sockets, similarly
42 to common network devices. The reasoning behind this is in depth
43 described in `Linux SocketCAN <https://www.kernel.org/doc/html/latest/networking/can.html>`_.
45 natural way to implement and work with higher layer protocols over CAN,
46 in the same way as, e.g., UDP/IP over Ethernet.
51 Before going into detail about the structure of a CAN bus device driver,
52 let's reiterate how the kernel gets to know about the device at all.
53 Some buses, like PCI or PCIe, support device enumeration. That is, when
54 the system boots, it discovers all the devices on the bus and reads
55 their configuration. The kernel identifies the device via its vendor ID
56 and device ID, and if there is a driver registered for this identifier
57 combination, its probe method is invoked to populate the driver's
58 instance for the given hardware. A similar situation goes with USB, only
59 it allows for device hot-plug.
61 The situation is different for peripherals which are directly embedded
62 in the SoC and connected to an internal system bus (AXI, APB, Avalon,
63 and others). These buses do not support enumeration, and thus the kernel
64 has to learn about the devices from elsewhere. This is exactly what the
65 Device Tree was made for.
70 An entry in device tree states that a device exists in the system, how
71 it is reachable (on which bus it resides) and its configuration –
72 registers address, interrupts and so on. An example of such a device
82 compatible = "simple-bus";
84 CTU_CAN_FD_0: CTU_CAN_FD@43c30000 {
85 compatible = "ctu,ctucanfd";
86 interrupt-parent = <&intc>;
87 interrupts = <0 30 4>;
89 reg = <0x43c30000 0x10000>;
95 .. _sec:socketcan:drv:
100 The driver can be divided into two parts – platform-dependent device
101 discovery and set up, and platform-independent CAN network device
104 .. _sec:socketcan:platdev:
106 Platform device driver
107 ^^^^^^^^^^^^^^^^^^^^^^
109 In the case of Zynq, the core is connected via the AXI system bus, which
110 does not have enumeration support, and the device must be specified in
111 Device Tree. This kind of devices is called *platform device* in the
112 kernel and is handled by a *platform device driver*\ [1]_.
114 A platform device driver provides the following things:
118 - A *remove* function
120 - A table of *compatible* devices that the driver can handle
122 The *probe* function is called exactly once when the device appears (or
123 the driver is loaded, whichever happens later). If there are more
124 devices handled by the same driver, the *probe* function is called for
125 each one of them. Its role is to allocate and initialize resources
126 required for handling the device, as well as set up low-level functions
127 for the platform-independent layer, e.g., *read_reg* and *write_reg*.
128 After that, the driver registers the device to a higher layer, in our
129 case as a *network device*.
131 The *remove* function is called when the device disappears, or the
132 driver is about to be unloaded. It serves to free the resources
133 allocated in *probe* and to unregister the device from higher layers.
135 Finally, the table of *compatible* devices states which devices the
136 driver can handle. The Device Tree entry ``compatible`` is matched
137 against the tables of all *platform drivers*.
141 /* Match table for OF platform binding */
142 static const struct of_device_id ctucan_of_match[] = {
143 { .compatible = "ctu,canfd-2", },
144 { .compatible = "ctu,ctucanfd", },
145 { /* end of list */ },
147 MODULE_DEVICE_TABLE(of, ctucan_of_match);
149 static int ctucan_probe(struct platform_device *pdev);
150 static int ctucan_remove(struct platform_device *pdev);
152 static struct platform_driver ctucanfd_driver = {
153 .probe = ctucan_probe,
154 .remove = ctucan_remove,
157 .of_match_table = ctucan_of_match,
160 module_platform_driver(ctucanfd_driver);
163 .. _sec:socketcan:netdev:
165 Network device driver
166 ^^^^^^^^^^^^^^^^^^^^^
168 Each network device must support at least these operations:
170 - Bring the device up: ``ndo_open``
172 - Bring the device down: ``ndo_close``
174 - Submit TX frames to the device: ``ndo_start_xmit``
176 - Signal TX completion and errors to the network subsystem: ISR
178 - Submit RX frames to the network subsystem: ISR and NAPI
180 There are two possible event sources: the device and the network
181 subsystem. Device events are usually signaled via an interrupt, handled
182 in an Interrupt Service Routine (ISR). Handlers for the events
183 originating in the network subsystem are then specified in
184 ``struct net_device_ops``.
186 When the device is brought up, e.g., by calling ``ip link set can0 up``,
187 the driver’s function ``ndo_open`` is called. It should validate the
188 interface configuration and configure and enable the device. The
189 analogous opposite is ``ndo_close``, called when the device is being
190 brought down, be it explicitly or implicitly.
192 When the system should transmit a frame, it does so by calling
193 ``ndo_start_xmit``, which enqueues the frame into the device. If the
194 device HW queue (FIFO, mailboxes or whatever the implementation is)
195 becomes full, the ``ndo_start_xmit`` implementation informs the network
196 subsystem that it should stop the TX queue (via ``netif_stop_queue``).
197 It is then re-enabled later in ISR when the device has some space
198 available again and is able to enqueue another frame.
200 All the device events are handled in ISR, namely:
202 #. **TX completion**. When the device successfully finishes transmitting
203 a frame, the frame is echoed locally. On error, an informative error
204 frame [2]_ is sent to the network subsystem instead. In both cases,
205 the software TX queue is resumed so that more frames may be sent.
207 #. **Error condition**. If something goes wrong (e.g., the device goes
208 bus-off or RX overrun happens), error counters are updated, and
209 informative error frames are enqueued to SW RX queue.
211 #. **RX buffer not empty**. In this case, read the RX frames and enqueue
212 them to SW RX queue. Usually NAPI is used as a middle layer (see ).
214 .. _sec:socketcan:napi:
219 The frequency of incoming frames can be high and the overhead to invoke
220 the interrupt service routine for each frame can cause significant
221 system load. There are multiple mechanisms in the Linux kernel to deal
222 with this situation. They evolved over the years of Linux kernel
223 development and enhancements. For network devices, the current standard
224 is NAPI – *the New API*. It is similar to classical top-half/bottom-half
225 interrupt handling in that it only acknowledges the interrupt in the ISR
226 and signals that the rest of the processing should be done in softirq
227 context. On top of that, it offers the possibility to *poll* for new
228 frames for a while. This has a potential to avoid the costly round of
229 enabling interrupts, handling an incoming IRQ in ISR, re-enabling the
230 softirq and switching context back to softirq.
232 See :ref:`Documentation/networking/napi.rst <napi>` for more information.
234 Integrating the core to Xilinx Zynq
235 -----------------------------------
237 The core interfaces a simple subset of the Avalon
238 (search for Intel **Avalon Interface Specifications**)
239 bus as it was originally used on
240 Alterra FPGA chips, yet Xilinx natively interfaces with AXI
241 (search for ARM **AMBA AXI and ACE Protocol Specification AXI3,
242 AXI4, and AXI4-Lite, ACE and ACE-Lite**).
243 The most obvious solution would be to use
244 an Avalon/AXI bridge or implement some simple conversion entity.
245 However, the core’s interface is half-duplex with no handshake
246 signaling, whereas AXI is full duplex with two-way signaling. Moreover,
247 even AXI-Lite slave interface is quite resource-intensive, and the
248 flexibility and speed of AXI are not required for a CAN core.
250 Thus a much simpler bus was chosen – APB (Advanced Peripheral Bus)
251 (search for ARM **AMBA APB Protocol Specification**).
252 APB-AXI bridge is directly available in
253 Xilinx Vivado, and the interface adaptor entity is just a few simple
254 combinatorial assignments.
256 Finally, to be able to include the core in a block diagram as a custom
257 IP, the core, together with the APB interface, has been packaged as a
260 CTU CAN FD Driver design
261 ------------------------
263 The general structure of a CAN device driver has already been examined
264 in . The next paragraphs provide a more detailed description of the CTU
265 CAN FD core driver in particular.
270 The core is not intended to be used solely with SocketCAN, and thus it
271 is desirable to have an OS-independent low-level driver. This low-level
272 driver can then be used in implementations of OS driver or directly
273 either on bare metal or in a user-space application. Another advantage
274 is that if the hardware slightly changes, only the low-level driver
275 needs to be modified.
277 The code [3]_ is in part automatically generated and in part written
278 manually by the core author, with contributions of the thesis’ author.
279 The low-level driver supports operations such as: set bit timing, set
280 controller mode, enable/disable, read RX frame, write TX frame, and so
283 Configuring bit timing
284 ~~~~~~~~~~~~~~~~~~~~~~
286 On CAN, each bit is divided into four segments: SYNC, PROP, PHASE1, and
287 PHASE2. Their duration is expressed in multiples of a Time Quantum
288 (details in `CAN Specification, Version 2.0 <http://esd.cs.ucr.edu/webres/can20.pdf>`_, chapter 8).
290 bitrate, the durations of all the segments (and time quantum) must be
291 computed from the bitrate and Sample Point. This is performed
292 independently for both the Nominal bitrate and Data bitrate for CAN FD.
294 SocketCAN is fairly flexible and offers either highly customized
295 configuration by setting all the segment durations manually, or a
296 convenient configuration by setting just the bitrate and sample point
297 (and even that is chosen automatically per Bosch recommendation if not
298 specified). However, each CAN controller may have different base clock
299 frequency and different width of segment duration registers. The
300 algorithm thus needs the minimum and maximum values for the durations
301 (and clock prescaler) and tries to optimize the numbers to fit both the
302 constraints and the requested parameters.
306 struct can_bittiming_const {
307 char name[16]; /* Name of the CAN controller hardware */
308 __u32 tseg1_min; /* Time segment 1 = prop_seg + phase_seg1 */
310 __u32 tseg2_min; /* Time segment 2 = phase_seg2 */
312 __u32 sjw_max; /* Synchronisation jump width */
313 __u32 brp_min; /* Bit-rate prescaler */
319 [lst:can_bittiming_const]
321 A curious reader will notice that the durations of the segments PROP_SEG
322 and PHASE_SEG1 are not determined separately but rather combined and
323 then, by default, the resulting TSEG1 is evenly divided between PROP_SEG
324 and PHASE_SEG1. In practice, this has virtually no consequences as the
325 sample point is between PHASE_SEG1 and PHASE_SEG2. In CTU CAN FD,
326 however, the duration registers ``PROP`` and ``PH1`` have different
327 widths (6 and 7 bits, respectively), so the auto-computed values might
328 overflow the shorter register and must thus be redistributed among the
334 Frame reception is handled in NAPI queue, which is enabled from ISR when
335 the RXNE (RX FIFO Not Empty) bit is set. Frames are read one by one
336 until either no frame is left in the RX FIFO or the maximum work quota
337 has been reached for the NAPI poll run (see ). Each frame is then passed
338 to the network interface RX queue.
340 An incoming frame may be either a CAN 2.0 frame or a CAN FD frame. The
341 way to distinguish between these two in the kernel is to allocate either
342 ``struct can_frame`` or ``struct canfd_frame``, the two having different
343 sizes. In the controller, the information about the frame type is stored
344 in the first word of RX FIFO.
346 This brings us a chicken-egg problem: we want to allocate the ``skb``
347 for the frame, and only if it succeeds, fetch the frame from FIFO;
348 otherwise keep it there for later. But to be able to allocate the
349 correct ``skb``, we have to fetch the first work of FIFO. There are
350 several possible solutions:
352 #. Read the word, then allocate. If it fails, discard the rest of the
353 frame. When the system is low on memory, the situation is bad anyway.
355 #. Always allocate ``skb`` big enough for an FD frame beforehand. Then
356 tweak the ``skb`` internals to look like it has been allocated for
357 the smaller CAN 2.0 frame.
359 #. Add option to peek into the FIFO instead of consuming the word.
361 #. If the allocation fails, store the read word into driver’s data. On
362 the next try, use the stored word instead of reading it again.
364 Option 1 is simple enough, but not very satisfying if we could do
365 better. Option 2 is not acceptable, as it would require modifying the
366 private state of an integral kernel structure. The slightly higher
367 memory consumption is just a virtual cherry on top of the “cake”. Option
368 3 requires non-trivial HW changes and is not ideal from the HW point of
371 Option 4 seems like a good compromise, with its disadvantage being that
372 a partial frame may stay in the FIFO for a prolonged time. Nonetheless,
373 there may be just one owner of the RX FIFO, and thus no one else should
374 see the partial frame (disregarding some exotic debugging scenarios).
375 Basides, the driver resets the core on its initialization, so the
376 partial frame cannot be “adopted” either. In the end, option 4 was
379 .. _subsec:ctucanfd:rxtimestamp:
381 Timestamping RX frames
382 ^^^^^^^^^^^^^^^^^^^^^^
384 The CTU CAN FD core reports the exact timestamp when the frame has been
385 received. The timestamp is by default captured at the sample point of
386 the last bit of EOF but is configurable to be captured at the SOF bit.
387 The timestamp source is external to the core and may be up to 64 bits
388 wide. At the time of writing, passing the timestamp from kernel to
389 userspace is not yet implemented, but is planned in the future.
394 The CTU CAN FD core has 4 independent TX buffers, each with its own
395 state and priority. When the core wants to transmit, a TX buffer in
396 Ready state with the highest priority is selected.
398 The priorities are 3bit numbers in register TX_PRIORITY
399 (nibble-aligned). This should be flexible enough for most use cases.
400 SocketCAN, however, supports only one FIFO queue for outgoing
401 frames [6]_. The buffer priorities may be used to simulate the FIFO
402 behavior by assigning each buffer a distinct priority and *rotating* the
403 priorities after a frame transmission is completed.
405 In addition to priority rotation, the SW must maintain head and tail
406 pointers into the FIFO formed by the TX buffers to be able to determine
407 which buffer should be used for next frame (``txb_head``) and which
408 should be the first completed one (``txb_tail``). The actual buffer
409 indices are (obviously) modulo 4 (number of TX buffers), but the
410 pointers must be at least one bit wider to be able to distinguish
411 between FIFO full and FIFO empty – in this situation,
412 :math:`txb\_head \equiv txb\_tail\ (\textrm{mod}\ 4)`. An example of how
413 the FIFO is maintained, together with priority rotation, is depicted in
417 +------+---+---+---+---+
418 | TXB# | 0 | 1 | 2 | 3 |
419 +======+===+===+===+===+
420 | Seq | A | B | C | |
421 +------+---+---+---+---+
422 | Prio | 7 | 6 | 5 | 4 |
423 +------+---+---+---+---+
425 +------+---+---+---+---+
429 +------+---+---+---+---+
430 | TXB# | 0 | 1 | 2 | 3 |
431 +======+===+===+===+===+
433 +------+---+---+---+---+
434 | Prio | 4 | 7 | 6 | 5 |
435 +------+---+---+---+---+
437 +------+---+---+---+---+
441 +------+---+---+---+---+----+
442 | TXB# | 0 | 1 | 2 | 3 | 0’ |
443 +======+===+===+===+===+====+
444 | Seq | E | B | C | D | |
445 +------+---+---+---+---+----+
446 | Prio | 4 | 7 | 6 | 5 | |
447 +------+---+---+---+---+----+
449 +------+---+---+---+---+----+
453 .. kernel-figure:: fsm_txt_buffer_user.svg
455 TX Buffer states with possible transitions
457 .. _subsec:ctucanfd:txtimestamp:
459 Timestamping TX frames
460 ^^^^^^^^^^^^^^^^^^^^^^
462 When submitting a frame to a TX buffer, one may specify the timestamp at
463 which the frame should be transmitted. The frame transmission may start
464 later, but not sooner. Note that the timestamp does not participate in
465 buffer prioritization – that is decided solely by the mechanism
468 Support for time-based packet transmission was recently merged to Linux
469 v4.19 `Time-based packet transmission <https://lwn.net/Articles/748879/>`_,
470 but it remains yet to be researched
471 whether this functionality will be practical for CAN.
473 Also similarly to retrieving the timestamp of RX frames, the core
474 supports retrieving the timestamp of TX frames – that is the time when
475 the frame was successfully delivered. The particulars are very similar
476 to timestamping RX frames and are described in .
478 Handling RX buffer overrun
479 ~~~~~~~~~~~~~~~~~~~~~~~~~~
481 When a received frame does no more fit into the hardware RX FIFO in its
482 entirety, RX FIFO overrun flag (STATUS[DOR]) is set and Data Overrun
483 Interrupt (DOI) is triggered. When servicing the interrupt, care must be
484 taken first to clear the DOR flag (via COMMAND[CDO]) and after that
485 clear the DOI interrupt flag. Otherwise, the interrupt would be
486 immediately [7]_ rearmed.
488 **Note**: During development, it was discussed whether the internal HW
489 pipelining cannot disrupt this clear sequence and whether an additional
490 dummy cycle is necessary between clearing the flag and the interrupt. On
491 the Avalon interface, it indeed proved to be the case, but APB being
492 safe because it uses 2-cycle transactions. Essentially, the DOR flag
493 would be cleared, but DOI register’s Preset input would still be high
494 the cycle when the DOI clear request would also be applied (by setting
495 the register’s Reset input high). As Set had higher priority than Reset,
496 the DOI flag would not be reset. This has been already fixed by swapping
497 the Set/Reset priority (see issue #187).
499 Reporting Error Passive and Bus Off conditions
500 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
502 It may be desirable to report when the node reaches *Error Passive*,
503 *Error Warning*, and *Bus Off* conditions. The driver is notified about
504 error state change by an interrupt (EPI, EWLI), and then proceeds to
505 determine the core’s error state by reading its error counters.
507 There is, however, a slight race condition here – there is a delay
508 between the time when the state transition occurs (and the interrupt is
509 triggered) and when the error counters are read. When EPI is received,
510 the node may be either *Error Passive* or *Bus Off*. If the node goes
511 *Bus Off*, it obviously remains in the state until it is reset.
512 Otherwise, the node is *or was* *Error Passive*. However, it may happen
513 that the read state is *Error Warning* or even *Error Active*. It may be
514 unclear whether and what exactly to report in that case, but I
515 personally entertain the idea that the past error condition should still
516 be reported. Similarly, when EWLI is received but the state is later
517 detected to be *Error Passive*, *Error Passive* should be reported.
520 CTU CAN FD Driver Sources Reference
521 -----------------------------------
523 .. kernel-doc:: drivers/net/can/ctucanfd/ctucanfd.h
526 .. kernel-doc:: drivers/net/can/ctucanfd/ctucanfd_base.c
529 .. kernel-doc:: drivers/net/can/ctucanfd/ctucanfd_pci.c
532 .. kernel-doc:: drivers/net/can/ctucanfd/ctucanfd_platform.c
535 CTU CAN FD IP Core and Driver Development Acknowledgment
536 ---------------------------------------------------------
538 * Odrej Ille <ondrej.ille@gmail.com>
540 * started the project as student at Department of Measurement, FEE, CTU
541 * invested great amount of personal time and enthusiasm to the project over years
542 * worked on more funded tasks
544 * `Department of Measurement <https://meas.fel.cvut.cz/>`_,
545 `Faculty of Electrical Engineering <http://www.fel.cvut.cz/en/>`_,
546 `Czech Technical University <https://www.cvut.cz/en>`_
548 * is the main investor into the project over many years
549 * uses project in their CAN/CAN FD diagnostics framework for `Skoda Auto <https://www.skoda-auto.cz/>`_
551 * `Digiteq Automotive <https://www.digiteqautomotive.com/en>`_
553 * funding of the project CAN FD Open Cores Support Linux Kernel Based Systems
554 * negotiated and paid CTU to allow public access to the project
555 * provided additional funding of the work
557 * `Department of Control Engineering <https://control.fel.cvut.cz/en>`_,
558 `Faculty of Electrical Engineering <http://www.fel.cvut.cz/en/>`_,
559 `Czech Technical University <https://www.cvut.cz/en>`_
561 * solving the project CAN FD Open Cores Support Linux Kernel Based Systems
562 * providing GitLab management
563 * virtual servers and computational power for continuous integration
564 * providing hardware for HIL continuous integration tests
566 * `PiKRON Ltd. <http://pikron.com/>`_
568 * minor funding to initiate preparation of the project open-sourcing
570 * Petr Porazil <porazil@pikron.com>
572 * design of PCIe transceiver addon board and assembly of boards
573 * design and assembly of MZ_APO baseboard for MicroZed/Zynq based system
575 * Martin Jerabek <martin.jerabek01@gmail.com>
577 * Linux driver development
578 * continuous integration platform architect and GHDL updates
579 * thesis `Open-source and Open-hardware CAN FD Protocol Support <https://dspace.cvut.cz/bitstream/handle/10467/80366/F3-DP-2019-Jerabek-Martin-Jerabek-thesis-2019-canfd.pdf>`_
581 * Jiri Novak <jnovak@fel.cvut.cz>
583 * project initiation, management and use at Department of Measurement, FEE, CTU
585 * Pavel Pisa <pisa@cmp.felk.cvut.cz>
587 * initiate open-sourcing, project coordination, management at Department of Control Engineering, FEE, CTU
589 * Jaroslav Beran<jara.beran@gmail.com>
591 * system integration for Intel SoC, core and driver testing and updates
593 * Carsten Emde (`OSADL <https://www.osadl.org/>`_)
595 * provided OSADL expertise to discuss IP core licensing
596 * pointed to possible deadlock for LGPL and CAN bus possible patent case which lead to relicense IP core design to BSD like license
598 * Reiner Zitzmann and Holger Zeltwanger (`CAN in Automation <https://www.can-cia.org/>`_)
600 * provided suggestions and help to inform community about the project and invited us to events focused on CAN bus future development directions
604 * implemented CTU CAN FD functional model for QEMU which has been integrated into QEMU mainline (`docs/system/devices/can.rst <https://www.qemu.org/docs/master/system/devices/can.html>`_)
605 * Bachelor thesis Model of CAN FD Communication Controller for QEMU Emulator
612 Other buses have their own specific driver interface to set up the
616 Not to be mistaken with CAN Error Frame. This is a ``can_frame`` with
617 ``CAN_ERR_FLAG`` set and some error info in its ``data`` field.
620 Available in CTU CAN FD repository
621 `<https://gitlab.fel.cvut.cz/canbus/ctucanfd_ip_core>`_
624 As is done in the low-level driver functions
625 ``ctucan_hw_set_nom_bittiming`` and
626 ``ctucan_hw_set_data_bittiming``.
629 At the time of writing this thesis, option 1 is still being used and
630 the modification is queued in gitlab issue #222
633 Strictly speaking, multiple CAN TX queues are supported since v4.19
634 `can: enable multi-queue for SocketCAN devices <https://lore.kernel.org/patchwork/patch/913526/>`_ but no mainline driver is using
638 Or rather in the next clock cycle