1 .. SPDX-License-Identifier: GPL-2.0
7 The SAS Layer is a management infrastructure which manages
8 SAS LLDDs. It sits between SCSI Core and SAS LLDDs. The
9 layout is as follows: while SCSI Core is concerned with
10 SAM/SPC issues, and a SAS LLDD+sequencer is concerned with
11 phy/OOB/link management, the SAS layer is concerned with:
13 * SAS Phy/Port/HA event management (LLDD generates,
15 * SAS Port management (creation/destruction),
16 * SAS Domain discovery and revalidation,
17 * SAS Domain device management,
18 * SCSI Host registration/unregistration,
19 * Device registration with SCSI Core (SAS) or libata
21 * Expander management and exporting expander control
24 A SAS LLDD is a PCI device driver. It is concerned with
25 phy/OOB management, and vendor specific tasks and generates
26 events to the SAS layer.
28 The SAS Layer does most SAS tasks as outlined in the SAS 1.1
31 The sas_ha_struct describes the SAS LLDD to the SAS layer.
32 Most of it is used by the SAS Layer but a few fields need to
33 be initialized by the LLDDs.
35 After initializing your hardware, from the probe() function
36 you call sas_register_ha(). It will register your LLDD with
37 the SCSI subsystem, creating a SCSI host and it will
38 register your SAS driver with the sysfs SAS tree it creates.
39 It will then return. Then you enable your phys to actually
40 start OOB (at which point your driver will start calling the
41 notify_* event callbacks).
43 Structure descriptions
44 ======================
49 Normally this is statically embedded to your driver's
54 struct sas_phy sas_phy;
58 And then all the phys are an array of my_phy in your HA
61 Then as you go along and initialize your phys you also
62 initialize the sas_phy struct, along with your own
65 In general, the phys are managed by the LLDD and the ports
66 are managed by the SAS layer. So the phys are initialized
67 and updated by the LLDD and the ports are initialized and
68 updated by the SAS layer.
70 There is a scheme where the LLDD can RW certain fields,
71 and the SAS layer can only read such ones, and vice versa.
72 The idea is to avoid unnecessary locking.
78 - must be set [0,MAX_PHYS)]
80 class, proto, type, role, oob_mode, linkrate
84 - you set this when OOB has finished and then notify
88 - this normally points to an array holding the sas
89 address of the phy, possibly somewhere in your my_phy
93 - set this when you (LLDD) receive an
94 IDENTIFY frame or a FIS frame, _before_ notifying the SAS
95 layer. The idea is that sometimes the LLDD may want to fake
96 or provide a different SAS address on that phy/port and this
97 allows it to do this. At best you should copy the sas
98 address from the IDENTIFY frame or maybe generate a SAS
99 address for SATA directly attached devices. The Discover
100 process may later change this.
103 - this is where you copy the IDENTIFY/FIS frame
104 when you get it; you lock, copy, set frame_rcvd_size and
105 unlock the lock, and then call the event. It is a pointer
106 since there's no way to know your hw frame size _exactly_,
107 so you define the actual array in your phy struct and let
108 this pointer point to it. You copy the frame from your
109 DMAable memory to that area holding the lock.
112 - this is where primitives go when they're
113 received. See sas.h. Grab the lock, set the primitive,
114 release the lock, notify.
117 - this points to the sas_port if the phy belongs
118 to a port -- the LLDD only reads this. It points to the
119 sas_port this phy is part of. Set by the SAS Layer.
122 - may be set; the SAS layer sets it anyway.
125 - you should set this to point to your phy so you
126 can find your way around faster when the SAS layer calls one
127 of your callbacks and passes you a phy. If the sas_phy is
128 embedded you can also use container_of -- whatever you
135 The LLDD doesn't set any fields of this struct -- it only
136 reads them. They should be self explanatory.
138 phy_mask is 32 bit, this should be enough for now, as I
139 haven't heard of a HA having more than 8 phys.
142 - I haven't found use for that -- maybe other
143 LLDD who wish to have internal port representation can make
146 ``struct sas_ha_struct``
147 ------------------------
149 It normally is statically declared in your own LLDD
150 structure describing your adapter::
154 struct sas_ha_struct sas_ha;
155 struct my_phy phys[MAX_PHYS];
156 struct sas_port sas_ports[MAX_PHYS]; /* (1) */
160 (1) If your LLDD doesn't have its own port representation.
162 What needs to be initialized (sample function given below).
168 - since the SAS layer doesn't want to mess with
169 memory allocation, etc, this points to statically
170 allocated array somewhere (say in your host adapter
171 structure) and holds the SAS address of the host
172 adapter as given by you or the manufacturer, etc.
178 - an array of pointers to structures. (see
179 note above on sas_addr).
180 These must be set. See more notes below.
183 - the number of phys present in the sas_phy array,
184 and the number of ports present in the sas_port
185 array. There can be a maximum num_phys ports (one per
186 port) so we drop the num_ports, and only use
189 The event interface::
191 /* LLDD calls these to notify the class of an event. */
192 void sas_notify_port_event(struct sas_phy *, enum port_event);
193 void sas_notify_phy_event(struct sas_phy *, enum phy_event);
194 void sas_notify_port_event_gfp(struct sas_phy *, enum port_event, gfp_t);
195 void sas_notify_phy_event_gfp(struct sas_phy *, enum phy_event, gfp_t);
197 The port notification::
199 /* The class calls these to notify the LLDD of an event. */
200 void (*lldd_port_formed)(struct sas_phy *);
201 void (*lldd_port_deformed)(struct sas_phy *);
203 If the LLDD wants notification when a port has been formed
204 or deformed it sets those to a function satisfying the type.
206 A SAS LLDD should also implement at least one of the Task
207 Management Functions (TMFs) described in SAM::
209 /* Task Management Functions. Must be called from process context. */
210 int (*lldd_abort_task)(struct sas_task *);
211 int (*lldd_abort_task_set)(struct domain_device *, u8 *lun);
212 int (*lldd_clear_aca)(struct domain_device *, u8 *lun);
213 int (*lldd_clear_task_set)(struct domain_device *, u8 *lun);
214 int (*lldd_I_T_nexus_reset)(struct domain_device *);
215 int (*lldd_lu_reset)(struct domain_device *, u8 *lun);
216 int (*lldd_query_task)(struct sas_task *);
218 For more information please read SAM from T10.org.
220 Port and Adapter management::
222 /* Port and Adapter management */
223 int (*lldd_clear_nexus_port)(struct sas_port *);
224 int (*lldd_clear_nexus_ha)(struct sas_ha_struct *);
226 A SAS LLDD should implement at least one of those.
231 int (*lldd_control_phy)(struct sas_phy *, enum phy_func);
234 - set this to point to your HA struct. You can also
235 use container_of if you embedded it as shown above.
237 A sample initialization and registration function
238 can look like this (called last thing from probe())
239 *but* before you enable the phys to do OOB::
241 static int register_sas_ha(struct my_sas_ha *my_ha)
244 static struct sas_phy *sas_phys[MAX_PHYS];
245 static struct sas_port *sas_ports[MAX_PHYS];
247 my_ha->sas_ha.sas_addr = &my_ha->sas_addr[0];
249 for (i = 0; i < MAX_PHYS; i++) {
250 sas_phys[i] = &my_ha->phys[i].sas_phy;
251 sas_ports[i] = &my_ha->sas_ports[i];
254 my_ha->sas_ha.sas_phy = sas_phys;
255 my_ha->sas_ha.sas_port = sas_ports;
256 my_ha->sas_ha.num_phys = MAX_PHYS;
258 my_ha->sas_ha.lldd_port_formed = my_port_formed;
260 my_ha->sas_ha.lldd_dev_found = my_dev_found;
261 my_ha->sas_ha.lldd_dev_gone = my_dev_gone;
263 my_ha->sas_ha.lldd_execute_task = my_execute_task;
265 my_ha->sas_ha.lldd_abort_task = my_abort_task;
266 my_ha->sas_ha.lldd_abort_task_set = my_abort_task_set;
267 my_ha->sas_ha.lldd_clear_aca = my_clear_aca;
268 my_ha->sas_ha.lldd_clear_task_set = my_clear_task_set;
269 my_ha->sas_ha.lldd_I_T_nexus_reset= NULL; (2)
270 my_ha->sas_ha.lldd_lu_reset = my_lu_reset;
271 my_ha->sas_ha.lldd_query_task = my_query_task;
273 my_ha->sas_ha.lldd_clear_nexus_port = my_clear_nexus_port;
274 my_ha->sas_ha.lldd_clear_nexus_ha = my_clear_nexus_ha;
276 my_ha->sas_ha.lldd_control_phy = my_control_phy;
278 return sas_register_ha(&my_ha->sas_ha);
281 (2) SAS 1.1 does not define I_T Nexus Reset TMF.
286 Events are **the only way** a SAS LLDD notifies the SAS layer
287 of anything. There is no other method or way a LLDD to tell
288 the SAS layer of anything happening internally or in the SAS
293 PHYE_LOSS_OF_SIGNAL, (C)
298 Port events, passed on a _phy_::
300 PORTE_BYTES_DMAED, (M)
301 PORTE_BROADCAST_RCVD, (E)
302 PORTE_LINK_RESET_ERR, (C)
303 PORTE_TIMER_EVENT, (C)
309 A SAS LLDD should be able to generate
311 - at least one event from group C (choice),
312 - events marked M (mandatory) are mandatory (only one),
313 - events marked E (expander) if it wants the SAS layer
314 to handle domain revalidation (only one such).
315 - Unmarked events are optional.
320 - when your HA got internal error and was reset.
323 - on receiving an IDENTIFY/FIS frame
326 - on receiving a primitive
329 - timer expired, loss of signal, loss of DWS, etc. [1]_
332 - DWS reset timeout timer expired [1]_
335 - Hard Reset primitive received.
338 - the device is gone [1]_
341 - OOB went fine and oob_mode is valid
344 - Error while doing OOB, the device probably
345 got disconnected. [1]_
348 - SATA is present, COMWAKE not sent.
350 .. [1] should set/clear the appropriate fields in the phy,
351 or alternatively call the inlined sas_phy_disconnected()
352 which is just a helper, from their tasklet.
354 The Execute Command SCSI RPC::
356 int (*lldd_execute_task)(struct sas_task *, gfp_t gfp_flags);
358 Used to queue a task to the SAS LLDD. @task is the task to be executed.
359 @gfp_mask is the gfp_mask defining the context of the caller.
361 This function should implement the Execute Command SCSI RPC,
363 That is, when lldd_execute_task() is called, the command
364 go out on the transport *immediately*. There is *no*
365 queuing of any sort and at any level in a SAS LLDD.
369 * -SAS_QUEUE_FULL, -ENOMEM, nothing was queued;
370 * 0, the task(s) were queued.
375 dev -- the device this task is destined to
376 task_proto -- _one_ of enum sas_proto
377 scatter -- pointer to scatter gather list array
378 num_scatter -- number of elements in scatter
379 total_xfer_len -- total number of bytes expected to be transferred
380 data_dir -- PCI_DMA_...
381 task_done -- callback when the task has finished execution
387 The sysfs tree has the following purposes:
389 a) It shows you the physical layout of the SAS domain at
390 the current time, i.e. how the domain looks in the
391 physical world right now.
392 b) Shows some device parameters _at_discovery_time_.
394 This is a link to the tree(1) program, very useful in
395 viewing the SAS domain:
396 ftp://mama.indstate.edu/linux/tree/
398 I expect user space applications to actually create a
399 graphical interface of this.
401 That is, the sysfs domain tree doesn't show or keep state if
402 you e.g., change the meaning of the READY LED MEANING
403 setting, but it does show you the current connection status
404 of the domain device.
406 Keeping internal device state changes is responsibility of
407 upper layers (Command set drivers) and user space.
409 When a device or devices are unplugged from the domain, this
410 is reflected in the sysfs tree immediately, and the device(s)
411 removed from the system.
413 The structure domain_device describes any device in the SAS
414 domain. It is completely managed by the SAS layer. A task
415 points to a domain device, this is how the SAS LLDD knows
416 where to send the task(s) to. A SAS LLDD only reads the
417 contents of the domain_device structure, but it never creates
420 Expander management from User Space
421 ===================================
423 In each expander directory in sysfs, there is a file called
424 "smp_portal". It is a binary sysfs attribute file, which
425 implements an SMP portal (Note: this is *NOT* an SMP port),
426 to which user space applications can send SMP requests and
427 receive SMP responses.
429 Functionality is deceptively simple:
431 1. Build the SMP frame you want to send. The format and layout
432 is described in the SAS spec. Leave the CRC field equal 0.
436 2. Open the expander's SMP portal sysfs file in RW mode.
440 3. Write the frame you built in 1.
444 4. Read the amount of data you expect to receive for the frame you built.
445 If you receive different amount of data you expected to receive,
446 then there was some kind of error.
450 All this process is shown in detail in the function do_smp_func()
451 and its callers, in the file "expander_conf.c".
453 The kernel functionality is implemented in the file
456 The program "expander_conf.c" implements this. It takes one
457 argument, the sysfs file name of the SMP portal to the
458 expander, and gives expander information, including routing
461 The SMP portal gives you complete control of the expander,
462 so please be careful.