[mirror_ubuntu-bionic-kernel.git] / Documentation / scsi / osd.txt

The OSD Standard
================
OSD (Object-Based Storage Device) is a T10 SCSI command set that is designed
to provide efficient operation of input/output logical units that manage the
allocation, placement, and accessing of variable-size data-storage containers,
called objects. Objects are intended to contain operating system and application
constructs. Each object has associated attributes attached to it, which are
integral part of the object and provide metadata about the object. The standard
defines some common obligatory attributes, but user attributes can be added as
needed.

See: http://www.t10.org/ftp/t10/drafts/osd2/ for the latest draft for OSD 2
or search the web for "OSD SCSI"

OSD in the Linux Kernel
=======================
osd-initiator:
  The main component of OSD in Kernel is the osd-initiator library. Its main
user is intended to be the pNFS-over-objects layout driver, which uses objects
as its back-end data storage. Other clients are the other osd parts listed below.

osd-uld:
  This is a SCSI ULD that registers for OSD type devices and provides a testing
platform, both for the in-kernel initiator as well as connected targets. It
currently has no useful user-mode API, though it could have if need be.

exofs:
  Is an OSD based Linux file system. It uses the osd-initiator and osd-uld,
to export a usable file system for users.
See Documentation/filesystems/exofs.txt for more details

osd target:
  There are no current plans for an OSD target implementation in kernel. For all
needs, a user-mode target that is based on the scsi tgt target framework is
available from Ohio Supercomputer Center (OSC) at:
http://www.open-osd.org/bin/view/Main/OscOsdProject
There are several other target implementations. See http://open-osd.org for more
links.

Files and Folders
=================
This is the complete list of files included in this work:
include/scsi/
	osd_initiator.h   Main API for the initiator library
	osd_types.h	  Common OSD types
	osd_sec.h	  Security Manager API
	osd_protocol.h	  Wire definitions of the OSD standard protocol
	osd_attributes.h  Wire definitions of OSD attributes

drivers/scsi/osd/
	osd_initiator.c   OSD-Initiator library implementation
	osd_uld.c	  The OSD scsi ULD
	osd_ktest.{h,c}	  In-kernel test suite (called by osd_uld)
	osd_debug.h	  Some printk macros
	Makefile	  For both in-tree and out-of-tree compilation
	Kconfig		  Enables inclusion of the different pieces
	osd_test.c	  User-mode application to call the kernel tests

The OSD-Initiator Library
=========================
osd_initiator is a low level implementation of an osd initiator encoder.
But even though, it should be intuitive and easy to use. Perhaps over time an
higher lever will form that automates some of the more common recipes.

init/fini:
- osd_dev_init() associates a scsi_device with an osd_dev structure
  and initializes some global pools. This should be done once per scsi_device
  (OSD LUN). The osd_dev structure is needed for calling osd_start_request().

- osd_dev_fini() cleans up before a osd_dev/scsi_device destruction.

OSD commands encoding, execution, and decoding of results:

struct osd_request's is used to iteratively encode an OSD command and carry
its state throughout execution. Each request goes through these stages:

a. osd_start_request() allocates the request.

b. Any of the osd_req_* methods is used to encode a request of the specified
   type.

c. osd_req_add_{get,set}_attr_* may be called to add get/set attributes to the
   CDB. "List" or "Page" mode can be used exclusively. The attribute-list API
   can be called multiple times on the same request. However, only one
   attribute-page can be read, as mandated by the OSD standard.

d. osd_finalize_request() computes offsets into the data-in and data-out buffers
   and signs the request using the provided capability key and integrity-
   check parameters.

e. osd_execute_request() may be called to execute the request via the block
   layer and wait for its completion.  The request can be executed
   asynchronously by calling the block layer API directly.

f. After execution, osd_req_decode_sense() can be called to decode the request's
   sense information.

g. osd_req_decode_get_attr() may be called to retrieve osd_add_get_attr_list()
   values.

h. osd_end_request() must be called to deallocate the request and any resource
   associated with it. Note that osd_end_request cleans up the request at any
   stage and it must always be called after a successful osd_start_request().

osd_request's structure:

The OSD standard defines a complex structure of IO segments pointed to by
members in the CDB. Up to 3 segments can be deployed in the IN-Buffer and up to
4 in the OUT-Buffer. The ASCII illustration below depicts a secure-read with
associated get+set of attributes-lists. Other combinations very on the same
basic theme. From no-segments-used up to all-segments-used.

|________OSD-CDB__________|
|                         |
|read_len (offset=0)     -|---------\
|                         |         |
|get_attrs_list_length    |         |
|get_attrs_list_offset   -|----\    |
|                         |    |    |
|retrieved_attrs_alloc_len|    |    |
|retrieved_attrs_offset  -|----|----|-\
|                         |    |    | |
|set_attrs_list_length    |    |    | |
|set_attrs_list_offset   -|-\  |    | |
|                         | |  |    | |
|in_data_integ_offset    -|-|--|----|-|-\
|out_data_integ_offset   -|-|--|--\ | | |
\_________________________/ |  |  | | | |
                            |  |  | | | |
|_______OUT-BUFFER________| |  |  | | | |
|      Set attr list      |</  |  | | | |
|                         |    |  | | | |
|-------------------------|    |  | | | |
|   Get attr descriptors  |<---/  | | | |
|                         |       | | | |
|-------------------------|       | | | |
|    Out-data integrity   |<------/ | | |
|                         |         | | |
\_________________________/         | | |
                                    | | |
|________IN-BUFFER________|         | | |
|      In-Data read       |<--------/ | |
|                         |           | |
|-------------------------|           | |
|      Get attr list      |<----------/ |
|                         |             |
|-------------------------|             |
|    In-data integrity    |<------------/
|                         |
\_________________________/

A block device request can carry bidirectional payload by means of associating
a bidi_read request with a main write-request. Each in/out request is described
by a chain of BIOs associated with each request.
The CDB is of a SCSI VARLEN CDB format, as described by OSD standard.
The OSD standard also mandates alignment restrictions at start of each segment.

In the code, in struct osd_request, there are two _osd_io_info structures to
describe the IN/OUT buffers above, two BIOs for the data payload and up to five
_osd_req_data_segment structures to hold the different segments allocation and
information.

Important: We have chosen to disregard the assumption that a BIO-chain (and
the resulting sg-list) describes a linear memory buffer. Meaning only first and
last scatter chain can be incomplete and all the middle chains are of PAGE_SIZE.
For us, a scatter-gather-list, as its name implies and as used by the Networking
layer, is to describe a vector of buffers that will be transferred to/from the
wire. It works very well with current iSCSI transport. iSCSI is currently the
only deployed OSD transport. In the future we anticipate SAS and FC attached OSD
devices as well.

The OSD Testing ULD
===================
TODO: More user-mode control on tests.

Authors, Mailing list
=====================
Please communicate with us on any deployment of osd, whether using this code
or not.

Any problems, questions, bug reports, lonely OSD nights, please email:
   OSD Dev List <osd-dev@open-osd.org>

More up-to-date information can be found on:
http://open-osd.org

Boaz Harrosh <ooo@electrozaur.com>

References
==========
Weber, R., "SCSI Object-Based Storage Device Commands",
T10/1355-D ANSI/INCITS 400-2004,
http://www.t10.org/ftp/t10/drafts/osd/osd-r10.pdf

Weber, R., "SCSI Object-Based Storage Device Commands -2 (OSD-2)"
T10/1729-D, Working Draft, rev. 3
http://www.t10.org/ftp/t10/drafts/osd2/osd2r03.pdf
Commit	Line	Data
78e0c621 BH	1	The OSD Standard
	2	================
	3	OSD (Object-Based Storage Device) is a T10 SCSI command set that is designed
	4	to provide efficient operation of input/output logical units that manage the
	5	allocation, placement, and accessing of variable-size data-storage containers,
	6	called objects. Objects are intended to contain operating system and application
	7	constructs. Each object has associated attributes attached to it, which are
	8	integral part of the object and provide metadata about the object. The standard
	9	defines some common obligatory attributes, but user attributes can be added as
	10	needed.
	11
	12	See: http://www.t10.org/ftp/t10/drafts/osd2/ for the latest draft for OSD 2
	13	or search the web for "OSD SCSI"
	14
	15	OSD in the Linux Kernel
	16	=======================
	17	osd-initiator:
	18	The main component of OSD in Kernel is the osd-initiator library. Its main
	19	user is intended to be the pNFS-over-objects layout driver, which uses objects
	20	as its back-end data storage. Other clients are the other osd parts listed below.
	21
	22	osd-uld:
	23	This is a SCSI ULD that registers for OSD type devices and provides a testing
	24	platform, both for the in-kernel initiator as well as connected targets. It
	25	currently has no useful user-mode API, though it could have if need be.
	26
	27	exofs:
	28	Is an OSD based Linux file system. It uses the osd-initiator and osd-uld,
	29	to export a usable file system for users.
	30	See Documentation/filesystems/exofs.txt for more details
	31
	32	osd target:
	33	There are no current plans for an OSD target implementation in kernel. For all
	34	needs, a user-mode target that is based on the scsi tgt target framework is
	35	available from Ohio Supercomputer Center (OSC) at:
	36	http://www.open-osd.org/bin/view/Main/OscOsdProject
	37	There are several other target implementations. See http://open-osd.org for more
	38	links.
	39
	40	Files and Folders
	41	=================
	42	This is the complete list of files included in this work:
	43	include/scsi/
	44	osd_initiator.h Main API for the initiator library
	45	osd_types.h Common OSD types
	46	osd_sec.h Security Manager API
	47	osd_protocol.h Wire definitions of the OSD standard protocol
	48	osd_attributes.h Wire definitions of OSD attributes
	49
	50	drivers/scsi/osd/
	51	osd_initiator.c OSD-Initiator library implementation
	52	osd_uld.c The OSD scsi ULD
	53	osd_ktest.{h,c} In-kernel test suite (called by osd_uld)
	54	osd_debug.h Some printk macros
	55	Makefile For both in-tree and out-of-tree compilation
	56	Kconfig Enables inclusion of the different pieces
	57	osd_test.c User-mode application to call the kernel tests
	58
	59	The OSD-Initiator Library
	60	=========================
	61	osd_initiator is a low level implementation of an osd initiator encoder.
	62	But even though, it should be intuitive and easy to use. Perhaps over time an
	63	higher lever will form that automates some of the more common recipes.
	64
65	init/fini:
66	- osd_dev_init() associates a scsi_device with an osd_dev structure
67	and initializes some global pools. This should be done once per scsi_device
68	(OSD LUN). The osd_dev structure is needed for calling osd_start_request().
69
70	- osd_dev_fini() cleans up before a osd_dev/scsi_device destruction.
71
72	OSD commands encoding, execution, and decoding of results:
73
74	struct osd_request's is used to iteratively encode an OSD command and carry
75	its state throughout execution. Each request goes through these stages:
76
77	a. osd_start_request() allocates the request.
78
79	b. Any of the osd_req_* methods is used to encode a request of the specified
80	type.
81
82	c. osd_req_add_{get,set}_attr_* may be called to add get/set attributes to the
83	CDB. "List" or "Page" mode can be used exclusively. The attribute-list API
84	can be called multiple times on the same request. However, only one
85	attribute-page can be read, as mandated by the OSD standard.
86
87	d. osd_finalize_request() computes offsets into the data-in and data-out buffers
88	and signs the request using the provided capability key and integrity-
89	check parameters.
90
91	e. osd_execute_request() may be called to execute the request via the block
92	layer and wait for its completion. The request can be executed
93	asynchronously by calling the block layer API directly.
94
95	f. After execution, osd_req_decode_sense() can be called to decode the request's
96	sense information.
97
98	g. osd_req_decode_get_attr() may be called to retrieve osd_add_get_attr_list()
99	values.
100
101	h. osd_end_request() must be called to deallocate the request and any resource
102	associated with it. Note that osd_end_request cleans up the request at any
103	stage and it must always be called after a successful osd_start_request().
104
105	osd_request's structure:
106
107	The OSD standard defines a complex structure of IO segments pointed to by
108	members in the CDB. Up to 3 segments can be deployed in the IN-Buffer and up to
109	4 in the OUT-Buffer. The ASCII illustration below depicts a secure-read with
110	associated get+set of attributes-lists. Other combinations very on the same
111	basic theme. From no-segments-used up to all-segments-used.
112
113	\|________OSD-CDB__________\|
114	\| \|
115	\|read_len (offset=0) -\|---------\
116	\| \| \|
117	\|get_attrs_list_length \| \|
118	\|get_attrs_list_offset -\|----\ \|
119	\| \| \| \|
120	\|retrieved_attrs_alloc_len\| \| \|
121	\|retrieved_attrs_offset -\|----\|----\|-\
122	\| \| \| \| \|
123	\|set_attrs_list_length \| \| \| \|
124	\|set_attrs_list_offset -\|-\ \| \| \|
125	\| \| \| \| \| \|
126	\|in_data_integ_offset -\|-\|--\|----\|-\|-\
127	\|out_data_integ_offset -\|-\|--\|--\ \| \| \|
128	\_________________________/ \| \| \| \| \| \|
129	\| \| \| \| \| \|
130	\|_______OUT-BUFFER________\| \| \| \| \| \| \|
131	\| Set attr list \|</ \| \| \| \| \|
132	\| \| \| \| \| \| \|
133	\|-------------------------\| \| \| \| \| \|
134	\| Get attr descriptors \|<---/ \| \| \| \|
135	\| \| \| \| \| \|
136	\|-------------------------\| \| \| \| \|
137	\| Out-data integrity \|<------/ \| \| \|
138	\| \| \| \| \|
139	\_________________________/ \| \| \|
140	\| \| \|
141	\|________IN-BUFFER________\| \| \| \|
142	\| In-Data read \|<--------/ \| \|
143	\| \| \| \|
144	\|-------------------------\| \| \|
145	\| Get attr list \|<----------/ \|
146	\| \| \|
147	\|-------------------------\| \|
148	\| In-data integrity \|<------------/
149	\| \|
150	\_________________________/
151
152	A block device request can carry bidirectional payload by means of associating
153	a bidi_read request with a main write-request. Each in/out request is described
154	by a chain of BIOs associated with each request.
155	The CDB is of a SCSI VARLEN CDB format, as described by OSD standard.
156	The OSD standard also mandates alignment restrictions at start of each segment.
157
158	In the code, in struct osd_request, there are two _osd_io_info structures to
159	describe the IN/OUT buffers above, two BIOs for the data payload and up to five
160	_osd_req_data_segment structures to hold the different segments allocation and
161	information.
162
163	Important: We have chosen to disregard the assumption that a BIO-chain (and
164	the resulting sg-list) describes a linear memory buffer. Meaning only first and
165	last scatter chain can be incomplete and all the middle chains are of PAGE_SIZE.
166	For us, a scatter-gather-list, as its name implies and as used by the Networking
167	layer, is to describe a vector of buffers that will be transferred to/from the
168	wire. It works very well with current iSCSI transport. iSCSI is currently the
169	only deployed OSD transport. In the future we anticipate SAS and FC attached OSD
170	devices as well.
171
172	The OSD Testing ULD
173	===================
174	TODO: More user-mode control on tests.
175
176	Authors, Mailing list
177	=====================
178	Please communicate with us on any deployment of osd, whether using this code
179	or not.
180
181	Any problems, questions, bug reports, lonely OSD nights, please email:
182	OSD Dev List <osd-dev@open-osd.org>
183
184	More up-to-date information can be found on:
185	http://open-osd.org
186
1fa3a002	187	Boaz Harrosh <ooo@electrozaur.com>
78e0c621 BH	188
	189	References
	190	==========
	191	Weber, R., "SCSI Object-Based Storage Device Commands",
	192	T10/1355-D ANSI/INCITS 400-2004,
	193	http://www.t10.org/ftp/t10/drafts/osd/osd-r10.pdf
	194
	195	Weber, R., "SCSI Object-Based Storage Device Commands -2 (OSD-2)"
	196	T10/1729-D, Working Draft, rev. 3
	197	http://www.t10.org/ftp/t10/drafts/osd2/osd2r03.pdf