Whamcloud - gitweb
LUDOC-11 misc: remove pre-2.5 conditional text
[doc/manual.git] / SettingLustreProperties.xml
1 <?xml version='1.0' encoding='UTF-8'?><chapter xmlns="http://docbook.org/ns/docbook" xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:lang="en-US" xml:id="settinglustreproperties">
2   <title xml:id="settinglustreproperties.title">Setting Lustre Properties in a C Program (<literal>llapi</literal>)</title>
3   <para>This chapter describes the <literal>llapi</literal> library of commands used for setting Lustre file properties within a C program running in a cluster environment, such as a data processing or MPI application. The commands described in this chapter are:</para>
4   <itemizedlist>
5     <listitem>
6       <para><xref linkend="dbdoclet.50438215_30970"/></para>
7     </listitem>
8     <listitem>
9       <para><xref linkend="dbdoclet.50438215_50149"/></para>
10     </listitem>
11     <listitem>
12       <para><xref linkend="dbdoclet.50438215_86607"/></para>
13     </listitem>
14     <listitem>
15       <para><xref linkend="dbdoclet.50438215_12433"/></para>
16     </listitem>
17     <listitem>
18       <para><xref linkend="dbdoclet.50438215_15718"/></para>
19     </listitem>
20   </itemizedlist>
21   <note>
22     <para>Lustre programming interface man pages are found in the <literal>lustre/doc</literal> folder.</para>
23   </note>
24   <section xml:id="dbdoclet.50438215_30970">
25     <title>
26       <literal>llapi_file_create</literal>
27     </title>
28     <para>Use <literal>llapi_file_create</literal> to set Lustre properties for a new file.</para>
29     <section remap="h5">
30       <title>Synopsis</title>
31       <screen>#include &lt;lustre/lustreapi.h&gt;
32
33 int llapi_file_create(char *name, long stripe_size, int stripe_offset, int stripe_count, int stripe_pattern);
34 </screen>
35     </section>
36     <section remap="h5">
37       <title>Description</title>
38       <para>The <literal>llapi_file_create()</literal> function sets a file descriptor&apos;s Lustre
39         file system striping information. The file descriptor is then accessed with
40           <literal>open()</literal>.</para>
41       <informaltable frame="all">
42         <tgroup cols="2">
43           <colspec colname="c1" colwidth="50*"/>
44           <colspec colname="c2" colwidth="50*"/>
45           <thead>
46             <row>
47               <entry>
48                 <para><emphasis role="bold">Option</emphasis></para>
49               </entry>
50               <entry>
51                 <para><emphasis role="bold">Description</emphasis></para>
52               </entry>
53             </row>
54           </thead>
55           <tbody>
56             <row>
57               <entry>
58                 <para> <literal>llapi_file_create()</literal></para>
59               </entry>
60               <entry>
61                 <para>If the file already exists, this parameter returns to &apos;<literal>EEXIST</literal>&apos;. If the stripe parameters are invalid, this parameter returns to &apos;<literal>EINVAL</literal>&apos;.</para>
62               </entry>
63             </row>
64             <row>
65               <entry>
66                 <para> <literal>stripe_size</literal></para>
67               </entry>
68               <entry>
69                 <para>This value must be an even multiple of system page size, as shown by <literal>getpagesize()</literal>. The default Lustre stripe size is 4MB.</para>
70               </entry>
71             </row>
72             <row>
73               <entry>
74                 <para> <literal>stripe_offset</literal></para>
75               </entry>
76               <entry>
77                 <para>Indicates the starting OST for this file.</para>
78               </entry>
79             </row>
80             <row>
81               <entry>
82                 <para> <literal>stripe_count</literal></para>
83               </entry>
84               <entry>
85                 <para>Indicates the number of OSTs that this file will be striped across.</para>
86               </entry>
87             </row>
88             <row>
89               <entry>
90                 <para> <literal>stripe_pattern</literal></para>
91               </entry>
92               <entry>
93                 <para>Indicates the RAID pattern.</para>
94               </entry>
95             </row>
96           </tbody>
97         </tgroup>
98       </informaltable>
99       <note>
100         <para>Currently, only RAID 0 is supported. To use the system defaults, set these values: <literal>stripe_size</literal> = 0, <literal>stripe_offset</literal> = -1, <literal>stripe_count</literal> = 0, <literal>stripe_pattern</literal> = 0</para>
101       </note>
102     </section>
103     <section remap="h5">
104       <title>Examples</title>
105       <para>System default size is 4 MB.</para>
106       <screen>char *tfile = TESTFILE;
107 int stripe_size = 65536</screen>
108       <para>To start at default, run:</para>
109       <screen>int stripe_offset = -1</screen>
110       <para>To start at the default, run:</para>
111       <screen>int stripe_count = 1</screen>
112       <para>To set a single stripe for this example, run:</para>
113       <screen>int stripe_pattern = 0</screen>
114       <para>Currently, only RAID 0 is supported.</para>
115       <screen>int stripe_pattern = 0; 
116 int rc, fd; 
117 rc = llapi_file_create(tfile, stripe_size,stripe_offset, stripe_count,stripe_pattern);</screen>
118       <para>Result code is inverted, you may return with &apos;<literal>EINVAL</literal>&apos; or an ioctl error.</para>
119       <screen>if (rc) {
120 fprintf(stderr,&quot;llapi_file_create failed: %d (%s) 0, rc, strerror(-rc));return -1; }</screen>
121       <para><literal>llapi_file_create</literal> closes the file descriptor. You must re-open the descriptor. To do this, run:</para>
122       <screen>fd = open(tfile, O_CREAT | O_RDWR | O_LOV_DELAY_CREATE, 0644); if (fd &lt; 0) \ { 
123 fprintf(stderr, &quot;Can&apos;t open %s file: %s0, tfile,
124 str-
125 error(errno));
126 return -1;
127 }</screen>
128     </section>
129   </section>
130   <section xml:id="dbdoclet.50438215_50149">
131     <title>llapi_file_get_stripe</title>
132     <para>Use <literal>llapi_file_get_stripe</literal> to get striping information for a file or directory on a Lustre file system.</para>
133     <section remap="h5">
134       <title>Synopsis</title>
135       <screen>
136 #include &lt;lustre/lustreapi.h&gt;
137  
138 int llapi_file_get_stripe(const char *<emphasis>path</emphasis>, void *<emphasis>lum</emphasis>);</screen>
139     </section>
140     <section remap="h5">
141       <title>Description</title>
142       <para>The <literal>llapi_file_get_stripe()</literal> function returns striping information for a file or directory <emphasis>path</emphasis> in <emphasis>lum</emphasis> (which should point to a large enough memory region) in one of the following formats:</para>
143       <screen>struct lov_user_md_v1 {
144 __u32 lmm_magic;
145 __u32 lmm_pattern;
146 __u64 lmm_object_id;
147 __u64 lmm_object_seq;
148 __u32 lmm_stripe_size;
149 __u16 lmm_stripe_count;
150 __u16 lmm_stripe_offset;
151 struct lov_user_ost_data_v1 lmm_objects[0];
152 } __attribute__((packed));
153 struct lov_user_md_v3 {
154 __u32 lmm_magic;
155 __u32 lmm_pattern;
156 __u64 lmm_object_id;
157 __u64 lmm_object_seq;
158 __u32 lmm_stripe_size;
159 __u16 lmm_stripe_count;
160 __u16 lmm_stripe_offset;
161 char lmm_pool_name[LOV_MAXPOOLNAME];
162 struct lov_user_ost_data_v1 lmm_objects[0];
163 } __attribute__((packed));</screen>
164       <informaltable frame="all">
165         <tgroup cols="2">
166           <colspec colname="c1" colwidth="50*"/>
167           <colspec colname="c2" colwidth="50*"/>
168           <thead>
169             <row>
170               <entry>
171                 <para><emphasis role="bold">Option</emphasis></para>
172               </entry>
173               <entry>
174                 <para><emphasis role="bold">Description</emphasis></para>
175               </entry>
176             </row>
177           </thead>
178           <tbody>
179             <row>
180               <entry>
181                 <para> <literal>lmm_magic</literal></para>
182               </entry>
183               <entry>
184                 <para>Specifies the format of the returned striping information. <literal>LOV_MAGIC_V1</literal> is used for lov_user_md_v1. LOV_MAGIC_V3 is used for <literal>lov_user_md_v3</literal>.</para>
185               </entry>
186             </row>
187             <row>
188               <entry>
189                 <para> <literal>lmm_pattern</literal></para>
190               </entry>
191               <entry>
192                 <para>Holds the striping pattern. Only <literal>LOV_PATTERN_RAID0</literal> is
193                   possible in this Lustre software release.</para>
194               </entry>
195             </row>
196             <row>
197               <entry>
198                 <para> <literal>lmm_object_id</literal></para>
199               </entry>
200               <entry>
201                 <para>Holds the MDS object ID.</para>
202               </entry>
203             </row>
204             <row>
205               <entry>
206                 <para> <literal>lmm_object_gr</literal></para>
207               </entry>
208               <entry>
209                 <para>Holds the MDS object group.</para>
210               </entry>
211             </row>
212             <row>
213               <entry>
214                 <para> <literal>lmm_stripe_size</literal></para>
215               </entry>
216               <entry>
217                 <para>Holds the stripe size in bytes.</para>
218               </entry>
219             </row>
220             <row>
221               <entry>
222                 <para> <literal>lmm_stripe_count</literal></para>
223               </entry>
224               <entry>
225                 <para>Holds the number of OSTs over which the file is striped.</para>
226               </entry>
227             </row>
228             <row>
229               <entry>
230                 <para> <literal>lmm_stripe_offset</literal></para>
231               </entry>
232               <entry>
233                 <para>Holds the OST index from which the file starts.</para>
234               </entry>
235             </row>
236             <row>
237               <entry>
238                 <para> <literal>lmm_pool_name</literal></para>
239               </entry>
240               <entry>
241                 <para>Holds the OST pool name to which the file belongs.</para>
242               </entry>
243             </row>
244             <row>
245               <entry>
246                 <para> <literal>lmm_objects</literal></para>
247               </entry>
248               <entry>
249                 <para>An array of <literal>lmm_stripe_count</literal> members containing per OST file information in</para>
250                 <para>the following format:</para>
251                 <screen>struct lov_user_ost_data_v1 {
252                 __u64 l_object_id;
253                 __u64 l_object_seq;
254                 __u32 l_ost_gen;
255                 __u32 l_ost_idx;
256                 } __attribute__((packed));</screen>
257               </entry>
258             </row>
259             <row>
260               <entry>
261                 <para> <literal>l_object_id</literal></para>
262               </entry>
263               <entry>
264                 <para>Holds the OST&apos;s object ID.</para>
265               </entry>
266             </row>
267             <row>
268               <entry>
269                 <para> <literal>l_object_seq</literal></para>
270               </entry>
271               <entry>
272                 <para>Holds the OST&apos;s object group.</para>
273               </entry>
274             </row>
275             <row>
276               <entry>
277                 <para> <literal>l_ost_gen</literal></para>
278               </entry>
279               <entry>
280                 <para>Holds the OST&apos;s index generation.</para>
281               </entry>
282             </row>
283             <row>
284               <entry>
285                 <para> <literal>l_ost_idx</literal></para>
286               </entry>
287               <entry>
288                 <para>Holds the OST&apos;s index in LOV.</para>
289               </entry>
290             </row>
291           </tbody>
292         </tgroup>
293       </informaltable>
294     </section>
295     <section remap="h5">
296       <title>Return Values</title>
297       <para><literal>llapi_file_get_stripe()</literal> returns:</para>
298       <para><literal>0</literal> On success</para>
299       <para><literal>!= 0</literal> On failure, <literal>errno</literal> is set appropriately</para>
300     </section>
301     <section remap="h5">
302       <title>Errors</title>
303       <informaltable frame="all">
304         <tgroup cols="2">
305           <colspec colname="c1" colwidth="50*"/>
306           <colspec colname="c2" colwidth="50*"/>
307           <thead>
308             <row>
309               <entry>
310                 <para><emphasis role="bold">Errors</emphasis></para>
311               </entry>
312               <entry>
313                 <para><emphasis role="bold">Description</emphasis></para>
314               </entry>
315             </row>
316           </thead>
317           <tbody>
318             <row>
319               <entry>
320                 <para> <literal>ENOMEM</literal></para>
321               </entry>
322               <entry>
323                 <para>Failed to allocate memory</para>
324               </entry>
325             </row>
326             <row>
327               <entry>
328                 <para> <literal>ENAMETOOLONG</literal></para>
329               </entry>
330               <entry>
331                 <para>Path was too long</para>
332               </entry>
333             </row>
334             <row>
335               <entry>
336                 <para> <literal>ENOENT</literal></para>
337               </entry>
338               <entry>
339                 <para>Path does not point to a file or directory</para>
340               </entry>
341             </row>
342             <row>
343               <entry>
344                 <para> <literal>ENOTTY</literal></para>
345               </entry>
346               <entry>
347                 <para>Path does not point to a Lustre file system</para>
348               </entry>
349             </row>
350             <row>
351               <entry>
352                 <para> <literal>EFAULT</literal></para>
353               </entry>
354               <entry>
355                 <para>Memory region pointed by lum is not properly mapped</para>
356               </entry>
357             </row>
358           </tbody>
359         </tgroup>
360       </informaltable>
361     </section>
362     <section remap="h5">
363       <title>Examples</title>
364       <programlisting>
365 #include &lt;stdio.h&gt;
366 #include &lt;stdlib.h&gt;
367 #include &lt;errno.h&gt;
368 #include &lt;lustre/lustreapi.h&gt;
369
370 static inline int maxint(int a, int b)
371 {
372         return a &gt; b ? a : b;
373 }
374 static void *alloc_lum()
375 {
376         int v1, v3, join;
377         v1 = sizeof(struct lov_user_md_v1) +
378                 LOV_MAX_STRIPE_COUNT * sizeof(struct lov_user_ost_data_v1);
379         v3 = sizeof(struct lov_user_md_v3) +
380                 LOV_MAX_STRIPE_COUNT * sizeof(struct lov_user_ost_data_v1);
381         return malloc(maxint(v1, v3));
382 }
383 int main(int argc, char** argv)
384 {
385         struct lov_user_md *lum_file = NULL;
386         int rc;
387         int lum_size;
388         if (argc != 2) {
389                 fprintf(stderr, &quot;Usage: %s &lt;filename&gt;\n&quot;, argv[0]);
390                 return 1;
391         }
392         lum_file = alloc_lum();
393         if (lum_file == NULL) {
394                 rc = ENOMEM;
395                 goto cleanup;
396         }
397         rc = llapi_file_get_stripe(argv[1], lum_file);
398         if (rc) {
399                 rc = errno;
400                 goto cleanup;
401         }
402         /* stripe_size stripe_count */
403         printf(&quot;%d %d\n&quot;,
404                         lum_file-&gt;lmm_stripe_size,
405                         lum_file-&gt;lmm_stripe_count);
406 cleanup:
407         if (lum_file != NULL)
408                 free(lum_file);
409         return rc;
410 }
411 </programlisting>
412     </section>
413   </section>
414   <section xml:id="dbdoclet.50438215_86607">
415     <title>
416       <literal>llapi_file_open</literal>
417     </title>
418     <para>The <literal>llapi_file_open</literal> command opens (or creates) a file or device on a
419       Lustre file system.</para>
420     <section remap="h5">
421       <title>Synopsis</title>
422       <screen>#include &lt;lustre/lustreapi.h&gt;
423 int llapi_file_open(const char *<emphasis>name</emphasis>, int <emphasis>flags</emphasis>, int <emphasis>mode</emphasis>, 
424    unsigned long long <emphasis>stripe_size</emphasis>, int <emphasis>stripe_offset</emphasis>, 
425    int <emphasis>stripe_count</emphasis>, int <emphasis>stripe_pattern</emphasis>);
426 int llapi_file_create(const char *<emphasis>name</emphasis>, unsigned long long <emphasis>stripe_size</emphasis>, 
427    int <emphasis>stripe_offset</emphasis>, int <emphasis>stripe_count</emphasis>, 
428    int <emphasis>stripe_pattern</emphasis>);
429 </screen>
430     </section>
431     <section remap="h5">
432       <title>Description</title>
433       <para>The <literal>llapi_file_create()</literal> call is equivalent to the <literal>llapi_file_open</literal> call with <emphasis>flags</emphasis> equal to <literal>O_CREAT|O_WRONLY</literal> and <emphasis>mode</emphasis> equal to <literal>0644</literal>, followed by file close.</para>
434       <para><literal>llapi_file_open()</literal> opens a file with a given name on a Lustre file
435         system.</para>
436       <informaltable frame="all">
437         <tgroup cols="2">
438           <colspec colname="c1" colwidth="50*"/>
439           <colspec colname="c2" colwidth="50*"/>
440           <thead>
441             <row>
442               <entry>
443                 <para><emphasis role="bold">Option</emphasis></para>
444               </entry>
445               <entry>
446                 <para><emphasis role="bold">Description</emphasis></para>
447               </entry>
448             </row>
449           </thead>
450           <tbody>
451             <row>
452               <entry>
453                 <para> <literal>flags</literal></para>
454               </entry>
455               <entry>
456                 <para>Can be a combination of <literal>O_RDONLY</literal>, <literal>O_WRONLY</literal>, <literal>O_RDWR</literal>, <literal>O_CREAT</literal>, <literal>O_EXCL</literal>, <literal>O_NOCTTY</literal>, <literal>O_TRUNC</literal>, <literal>O_APPEND</literal>, <literal>O_NONBLOCK</literal>, <literal>O_SYNC</literal>, <literal>FASYNC</literal>, <literal>O_DIRECT</literal>, <literal>O_LARGEFILE</literal>, <literal>O_DIRECTORY</literal>, <literal>O_NOFOLLOW</literal>, <literal>O_NOATIME</literal>.</para>
457               </entry>
458             </row>
459             <row>
460               <entry>
461                 <para> <literal>mode</literal></para>
462               </entry>
463               <entry>
464                 <para>Specifies the permission bits to be used for a new file when <literal>O_CREAT</literal> is used.</para>
465               </entry>
466             </row>
467             <row>
468               <entry>
469                 <para> <literal>stripe_size</literal></para>
470               </entry>
471               <entry>
472                 <para>Specifies stripe size (in bytes). Should be multiple of 64 KB, not exceeding 4 GB.</para>
473               </entry>
474             </row>
475             <row>
476               <entry>
477                 <para> <literal>stripe_offset</literal></para>
478               </entry>
479               <entry>
480                 <para>Specifies an OST index from which the file should start. The default value is -1.</para>
481               </entry>
482             </row>
483             <row>
484               <entry>
485                 <para> <literal>stripe_count</literal></para>
486               </entry>
487               <entry>
488                 <para>Specifies the number of OSTs to stripe the file across. The default value is -1.</para>
489               </entry>
490             </row>
491             <row>
492               <entry>
493                 <para> <literal>stripe_pattern</literal></para>
494               </entry>
495               <entry>
496                 <para>Specifies the striping pattern. In this release of the Lustre software, only
497                     <literal>LOV_PATTERN_RAID0</literal> is available. The default value is
498                   0.</para>
499               </entry>
500             </row>
501           </tbody>
502         </tgroup>
503       </informaltable>
504     </section>
505     <section remap="h5">
506       <title>Return Values</title>
507       <para><literal>llapi_file_open()</literal> and <literal>llapi_file_create()</literal> return:</para>
508       <para><literal>&gt;=0</literal> On success, for <literal>llapi_file_open</literal> the return value is a file descriptor</para>
509       <para><literal>&lt;0</literal> On failure, the absolute value is an error code</para>
510     </section>
511     <section remap="h5">
512       <title>Errors</title>
513       <informaltable frame="all">
514         <tgroup cols="2">
515           <colspec colname="c1" colwidth="50*"/>
516           <colspec colname="c2" colwidth="50*"/>
517           <thead>
518             <row>
519               <entry>
520                 <para><emphasis role="bold">Errors</emphasis></para>
521               </entry>
522               <entry>
523                 <para><emphasis role="bold">Description</emphasis></para>
524               </entry>
525             </row>
526           </thead>
527           <tbody>
528             <row>
529               <entry>
530                 <para> <literal>EINVAL</literal></para>
531               </entry>
532               <entry>
533                 <para><literal>stripe_size</literal> or <literal>stripe_offset</literal> or <literal>stripe_count</literal> or <literal>stripe_pattern</literal> is invalid.</para>
534               </entry>
535             </row>
536             <row>
537               <entry>
538                 <para> <literal>EEXIST</literal></para>
539               </entry>
540               <entry>
541                 <para>Striping information has already been set and cannot be altered; <literal>name</literal> already exists.</para>
542               </entry>
543             </row>
544             <row>
545               <entry>
546                 <para> <literal>EALREADY</literal></para>
547               </entry>
548               <entry>
549                 <para>Striping information has already been set and cannot be altered</para>
550               </entry>
551             </row>
552             <row>
553               <entry>
554                 <para> <literal>ENOTTY</literal></para>
555               </entry>
556               <entry>
557                 <para>
558                   <literal>name</literal> may not point to a Lustre file system.</para>
559               </entry>
560             </row>
561           </tbody>
562         </tgroup>
563       </informaltable>
564     </section>
565     <section remap="h5">
566       <title>Example</title>
567       <programlisting>
568 #include &lt;stdio.h&gt;
569 #include &lt;lustre/lustreapi.h&gt;
570
571 int main(int argc, char *argv[])
572 {
573         int rc;
574         if (argc != 2)
575                 return -1;
576         rc = llapi_file_create(argv[1], 1048576, 0, 2, LOV_PATTERN_RAID0);
577         if (rc &lt; 0) {
578                 fprintf(stderr, &quot;file creation has failed, %s\n&quot;,         strerror(-rc));
579                 return -1;
580         }
581         printf(&quot;%s with stripe size 1048576, striped across 2 OSTs,&quot;
582                         &quot; has been created!\n&quot;, argv[1]);
583         return 0;
584 }
585 </programlisting>
586     </section>
587   </section>
588   <section xml:id="dbdoclet.50438215_12433">
589     <title>
590       <literal>llapi_quotactl</literal>
591     </title>
592     <para>Use <literal>llapi_quotact</literal>l to manipulate disk quotas on a Lustre file system.</para>
593     <section remap="h5">
594       <title>Synopsis</title>
595       <screen>#include &lt;lustre/lustreapi.h&gt;
596 int llapi_quotactl(char&quot; &quot; *mnt,&quot; &quot; struct if_quotactl&quot; &quot; *qctl)
597  
598 struct if_quotactl {
599         __u32                   qc_cmd;
600         __u32                   qc_type;
601         __u32                   qc_id;
602         __u32                   qc_stat;
603         struct obd_dqinfo       qc_dqinfo;
604         struct obd_dqblk        qc_dqblk;
605         char                    obd_type[16];
606         struct obd_uuid         obd_uuid;
607 };
608 struct obd_dqblk {
609         __u64 dqb_bhardlimit;
610         __u64 dqb_bsoftlimit;
611         __u64 dqb_curspace;
612         __u64 dqb_ihardlimit;
613         __u64 dqb_isoftlimit;
614         __u64 dqb_curinodes;
615         __u64 dqb_btime;
616         __u64 dqb_itime;
617         __u32 dqb_valid;
618         __u32 padding;
619 };
620 struct obd_dqinfo {
621         __u64 dqi_bgrace;
622         __u64 dqi_igrace;
623         __u32 dqi_flags;
624         __u32 dqi_valid;
625 };
626 struct obd_uuid {
627         char uuid[40];
628 };</screen>
629     </section>
630     <section remap="h5">
631       <title>Description</title>
632       <para>The <literal>llapi_quotactl()</literal> command manipulates disk quotas on a Lustre file system mount. qc_cmd indicates a command to be applied to UID <literal>qc_id</literal> or GID <literal>qc_id</literal>.</para>
633       <informaltable frame="all">
634         <tgroup cols="2">
635           <colspec colname="c1" colwidth="50*"/>
636           <colspec colname="c2" colwidth="50*"/>
637           <thead>
638             <row>
639               <entry>
640                 <para><emphasis role="bold">Option</emphasis></para>
641               </entry>
642               <entry>
643                 <para><emphasis role="bold">Description</emphasis></para>
644               </entry>
645             </row>
646           </thead>
647           <tbody>
648             <row>
649               <entry>
650                 <para> <literal>LUSTRE_Q_GETQUOTA</literal></para>
651               </entry>
652               <entry>
653                 <para>Gets disk quota limits and current usage for user or group <emphasis>qc_id</emphasis>. <emphasis>qc_type</emphasis> is <literal>USRQUOTA</literal> or <literal>GRPQUOTA</literal>. <emphasis>uuid</emphasis> may be filled with <literal>OBD UUID</literal> string to query quota information from a specific node. <emphasis>dqb_valid</emphasis> may be set nonzero to query information only from MDS. If <emphasis>uuid</emphasis> is an empty string and <emphasis>dqb_valid</emphasis> is zero then cluster-wide limits and usage are returned. On return, <emphasis>obd_dqblk</emphasis> contains the requested information (block limits unit is kilobyte). Quotas must be turned on before using this command.</para>
654               </entry>
655             </row>
656             <row>
657               <entry>
658                 <para> <literal>LUSTRE_Q_SETQUOTA</literal></para>
659               </entry>
660               <entry>
661                 <para>Sets disk quota limits for user or group <emphasis>qc_id</emphasis>. <emphasis>qc_type</emphasis> is <literal>USRQUOTA</literal> or <literal>GRPQUOTA</literal>. <emphasis>dqb_valid</emphasis> must be set to <literal>QIF_ILIMITS</literal>, <literal>QIF_BLIMITS</literal> or <literal>QIF_LIMITS</literal> (both inode limits and block limits) dependent on updating limits. <emphasis>obd_dqblk</emphasis> must be filled with limits values (as set in <emphasis>dqb_valid</emphasis>, block limits unit is kilobyte). Quotas must be turned on before using this command.</para>
662               </entry>
663             </row>
664             <row>
665               <entry>
666                 <para> <literal>LUSTRE_Q_GETINFO</literal></para>
667               </entry>
668               <entry>
669                 <para>Gets information about quotas. <emphasis>qc_type</emphasis> is either
670                     <literal>USRQUOTA</literal> or <literal>GRPQUOTA</literal>. On return,
671                     <emphasis>dqi_igrace</emphasis> is inode grace time (in seconds),
672                     <emphasis>dqi_bgrace</emphasis> is block grace time (in seconds),
673                     <emphasis>dqi_flags</emphasis> is not used by the current release of the Lustre
674                   software.</para>
675               </entry>
676             </row>
677             <row>
678               <entry>
679                 <para> <literal>LUSTRE_Q_SETINFO</literal></para>
680               </entry>
681               <entry>
682                 <para>Sets quota information (like grace times). <emphasis>qc_type</emphasis> is
683                   either <literal>USRQUOTA</literal> or <literal>GRPQUOTA</literal>.
684                     <emphasis>dqi_igrace</emphasis> is inode grace time (in seconds),
685                     <emphasis>dqi_bgrace</emphasis> is block grace time (in seconds),
686                     <emphasis>dqi_flags</emphasis> is not used by the current release of the Lustre
687                   software and must be zeroed.</para>
688               </entry>
689             </row>
690           </tbody>
691         </tgroup>
692       </informaltable>
693     </section>
694     <section remap="h5">
695       <title>Return Values</title>
696       <para><literal>llapi_quotactl()</literal> returns:</para>
697       <para><literal>0</literal> On success</para>
698       <para><literal> -1 </literal> On failure and sets error number (<literal>errno</literal>) to indicate the error</para>
699     </section>
700     <section remap="h5">
701       <title>Errors</title>
702       <para><literal>llapi_quotactl</literal> errors are described below.</para>
703       <informaltable frame="all">
704         <tgroup cols="2">
705           <colspec colname="c1" colwidth="50*"/>
706           <colspec colname="c2" colwidth="50*"/>
707           <thead>
708             <row>
709               <entry>
710                 <para><emphasis role="bold">Errors</emphasis></para>
711               </entry>
712               <entry>
713                 <para><emphasis role="bold">Description</emphasis></para>
714               </entry>
715             </row>
716           </thead>
717           <tbody>
718             <row>
719               <entry>
720                 <para> <literal>EFAULT</literal></para>
721               </entry>
722               <entry>
723                 <para><emphasis>qctl</emphasis> is invalid.</para>
724               </entry>
725             </row>
726             <row>
727               <entry>
728                 <para> <literal>ENOSYS</literal></para>
729               </entry>
730               <entry>
731                 <para>Kernel or Lustre modules have not been compiled with the <literal>QUOTA</literal> option.</para>
732               </entry>
733             </row>
734             <row>
735               <entry>
736                 <para> <literal>ENOMEM</literal></para>
737               </entry>
738               <entry>
739                 <para>Insufficient memory to complete operation.</para>
740               </entry>
741             </row>
742             <row>
743               <entry>
744                 <para> <literal>ENOTTY</literal></para>
745               </entry>
746               <entry>
747                 <para> <emphasis>qc_cmd</emphasis> is invalid.</para>
748               </entry>
749             </row>
750             <row>
751               <entry>
752                 <para> <literal>EBUSY</literal></para>
753               </entry>
754               <entry>
755                 <para>Cannot process during quotacheck.</para>
756               </entry>
757             </row>
758             <row>
759               <entry>
760                 <para> <literal>ENOENT</literal></para>
761               </entry>
762               <entry>
763                 <para> <emphasis>uuid</emphasis> does not correspond to OBD or <emphasis>mnt</emphasis> does not exist.</para>
764               </entry>
765             </row>
766             <row>
767               <entry>
768                 <para> <literal>EPERM</literal></para>
769               </entry>
770               <entry>
771                 <para>The call is privileged and the caller is not the super user.</para>
772               </entry>
773             </row>
774             <row>
775               <entry>
776                 <para> <literal>ESRCH</literal></para>
777               </entry>
778               <entry>
779                 <para>No disk quota is found for the indicated user. Quotas have not been turned on for this file system.</para>
780               </entry>
781             </row>
782           </tbody>
783         </tgroup>
784       </informaltable>
785     </section>
786   </section>
787   <section xml:id="dbdoclet.50438215_15718">
788     <title>
789       <literal>llapi_path2fid</literal>
790     </title>
791     <para>Use <literal>llapi_path2fid</literal> to get the FID from the pathname.</para>
792     <section remap="h5">
793       <title>Synopsis</title>
794       <screen>#include &lt;lustre/lustreapi.h&gt;
795  
796 int llapi_path2fid(const char *path, unsigned long long *seq, unsigned long *oid, unsigned long *ver)</screen>
797     </section>
798     <section remap="h5">
799       <title>Description</title>
800       <para>The <literal>llapi_path2fid</literal> function returns the FID (sequence : object ID : version) for the pathname.</para>
801     </section>
802     <section remap="h5">
803       <title>Return Values</title>
804       <para><literal>llapi_path2fid</literal> returns:</para>
805       <para><literal>0</literal> On success</para>
806       <para>non-zero value On failure</para>
807     </section>
808   </section>
809   <section condition="l29">
810       <title>
811           <literal>llapi_ladvise</literal>
812       </title>
813       <para>Use <literal>llapi_ladvise</literal> to give IO advice/hints on a
814       Lustre file to the server.</para>
815       <section remap="h5">
816           <title>Synopsis</title>
817           <screen>
818 #include &lt;lustre/lustreapi.h&gt;
819 int llapi_ladvise(int fd, unsigned long long flags,
820                   int num_advise, struct llapi_lu_ladvise *ladvise);
821                                 
822 struct llapi_lu_ladvise {
823   __u16 lla_advice;       /* advice type */
824   __u16 lla_value1;       /* values for different advice types */
825   __u32 lla_value2;
826   __u64 lla_start;        /* first byte of extent for advice */
827   __u64 lla_end;          /* last byte of extent for advice */
828   __u32 lla_value3;
829   __u32 lla_value4;
830 };
831           </screen>
832       </section>
833       <section remap="h5">
834           <title>Description</title>
835           <para>The <literal>llapi_ladvise</literal> function passes an array of
836           <emphasis>num_advise</emphasis> I/O hints (up to a maximum of
837           <emphasis>LAH_COUNT_MAX</emphasis> items) in ladvise for the file
838           descriptor <emphasis>fd</emphasis> from an application to one or more
839           Lustre servers.  Optionally, <emphasis>flags</emphasis> can modify how
840           the advice will be processed via bitwise-or'd values:</para>
841           <itemizedlist>
842           <listitem>
843           <para><literal>LF_ASYNC</literal>: Clients return to userspace
844           immediately after submitting ladvise RPCs, leaving server threads to
845           handle the advices asynchronously.</para>
846           </listitem>
847           <listitem>
848           <para><literal>LF_UNSET</literal>: Unset/clear a previous advice
849           (Currently only supports LU_ADVISE_LOCKNOEXPAND).</para>
850           </listitem>
851           </itemizedlist>
852           <para>Each of the <emphasis>ladvise</emphasis> elements is an
853           <emphasis>llapi_lu_ladvise</emphasis> structure, which contains the
854           following fields:
855           <informaltable frame="all">
856             <tgroup cols="2">
857               <colspec colname="c1" colwidth="50*"/>
858               <colspec colname="c2" colwidth="50*"/>
859               <thead>
860                 <row>
861                   <entry>
862                     <para><emphasis role="bold">Field</emphasis></para>
863                   </entry>
864                   <entry>
865                     <para><emphasis role="bold">Description</emphasis></para>
866                   </entry>
867                 </row>
868               </thead>
869               <tbody>
870               <row>
871               <entry>
872                 <para> <literal>lla_ladvice</literal></para>
873               </entry>
874               <entry>
875                 <para>Specifies the advice for the given file range, currently
876                 one of:</para>
877                 <para><literal>LU_LADVISE_WILLREAD</literal>:  Prefetch data
878                 into server cache using optimum I/O size for the server.</para>
879                 <para><literal>LU_LADVISE_DONTNEED</literal>:  Clean cached data
880                 for the specified file range(s) on the server.</para>
881               </entry>
882               </row>
883               <row>
884                 <entry>
885                   <para> <literal>lla_start</literal></para>
886                 </entry>
887                 <entry>
888                   <para>The offset in bytes for the start of this advice.</para>
889                 </entry>
890               </row>
891               <row>
892                 <entry>
893                   <para> <literal>lla_end</literal></para>
894                 </entry>
895                 <entry>
896                   <para>The offset in bytes (non-inclusive) for the end of this
897                   advice.</para>
898                 </entry>
899               </row>
900               <row>
901                 <entry>
902                   <para> <literal>lla_value1</literal></para>
903                   <para> <literal>lla_value2</literal></para>
904                   <para> <literal>lla_value3</literal></para>
905                   <para> <literal>lla_value4</literal></para>
906                 </entry>
907                 <entry>
908                     <para>Additional arguments for future advice types and
909                     should be set to zero if not explicitly required for a given
910                     advice type.  Advice-specific names for these fields
911                     follow.</para>
912                 </entry>
913               </row>
914               <row>
915                 <entry>
916                   <para> <literal>lla_lockahead_mode</literal></para>
917                 </entry>
918                 <entry>
919                   <para>When using LU_ADVISE_LOCKAHEAD, the 'lla_value1' field
920                   is used to communicate the requested lock mode, and can be
921                   referred to as lla_lockahead_mode.</para>
922                 </entry>
923               </row>
924               <row>
925                 <entry>
926                   <para> <literal>lla_peradvice_flags</literal></para>
927                 </entry>
928                 <entry>
929                   <para>When using advices which support them, the 'lla_value2'
930                   field is used to communicate per-advice flags and can be
931                   referred to as 'lla_peradvice_flags'. Both LF_ASYNC and
932                   LF_UNSET are supported as peradvice flags.</para>
933                 </entry>
934               </row>
935               <row>
936                 <entry>
937                   <para> <literal>lla_lockahead_result</literal></para>
938                 </entry>
939                 <entry>
940                   <para>When using LU_ADVISE_LOCKAHEAD, the 'lla_value3' field
941                   is used to communicate the result of the request, and can be
942                   referred to as lla_lockahead_result.</para>
943                 </entry>
944               </row>
945               </tbody>
946               </tgroup>
947           </informaltable>
948           </para>
949           <para><literal>llapi_ladvise()</literal> forwards the advice to Lustre
950           servers without guaranteeing how and when servers will react to the
951           advice. Actions may or may not be triggered when the advices are
952           received, depending on the type of the advice as well as the real-time
953           decision of the affected server-side components.
954           </para>
955           <para> A typical usage of <literal>llapi_ladvise()</literal> is to
956           enable applications and users (via <literal>lfs ladvise</literal>)
957           with external knowledge about application I/O patterns to intervene in
958           server-side I/O handling. For example, if a group of different clients
959           are doing small random reads of a file, prefetching pages into OSS
960           cache with big linear reads before the random IO is an overall net
961           benefit.  Fetching that data into each client cache with
962           <emphasis>fadvise()</emphasis> may not be beneficial, due to much more
963           data being sent to the clients.
964           </para>
965           <para>
966           LU_LADVISE_LOCKAHEAD merits a special comment.  While it is possible
967           and encouraged to use it directly in your application to avoid lock
968           contention (primarily for writing to a single file from multiple
969           clients), it will also be available in the MPI-I/O / MPICH library
970           from ANL for use with the i/o aggregation mode of that library.  This
971           is intended (eventually) to be the primary way this feature is used.
972           </para>
973           <para>
974           At the time of writing, this support is proposed as a patch but is
975           not yet merged in to the public ANL code base.  Users are encouraged
976           to check their MPICH documentation and/or check with their library
977           provider about support.
978           </para>
979           <para>While conceptually similar to the
980           <emphasis>posix_fadvise</emphasis> and Linux
981           <emphasis>fadvise</emphasis> system calls, the main difference of
982           <literal>llapi_ladvise()</literal> is that
983           <emphasis>fadvise() / posix_fadvise()</emphasis> are client side
984           mechanisms that do not pass advice to the filesystem, while
985           <literal>llapi_ladvise()</literal> sends advice or hints to one or
986           more Lustre servers on which the file is stored.  In some cases it may
987           be desirable to use both interfaces.
988           </para>
989       </section>
990       <section remap="h5">
991           <title>Return Values</title>
992           <para><literal>llapi_ladvise</literal> returns:</para>
993           <para><literal>0</literal> On success</para>
994           <para><literal>-1</literal> if an error occurred (in which case, errno
995           is set appropriately).</para>
996       </section>
997       <section remap="h5">
998           <title>Errors</title>
999         <para>
1000           <informaltable frame="all">
1001             <tgroup cols="2">
1002               <colspec colname="c1" colwidth="50*"/>
1003               <colspec colname="c2" colwidth="50*"/>
1004               <thead>
1005                 <row>
1006                   <entry>
1007                     <para><emphasis role="bold">Error</emphasis></para>
1008                   </entry>
1009                   <entry>
1010                     <para><emphasis role="bold">Description</emphasis></para>
1011                   </entry>
1012                 </row>
1013               </thead>
1014               <tbody>
1015                 <row>
1016                   <entry>
1017                     <para> <literal>ENOMEM</literal></para>
1018                   </entry>
1019                   <entry>
1020                     <para>Insufficient memory to complete operation.</para>
1021                   </entry>
1022                 </row>
1023                 <row>
1024                   <entry>
1025                     <para> <literal>EINVAL</literal></para>
1026                   </entry>
1027                   <entry>
1028                     <para>One or more invalid arguments are given.</para>
1029                   </entry>
1030                 </row>
1031                 <row>
1032                   <entry>
1033                     <para> <literal>EFAULT</literal></para>
1034                   </entry>
1035                   <entry>
1036                     <para>Memory region pointed by
1037                     <literal>ladvise</literal> is not properly mapped.
1038                     </para>
1039                   </entry>
1040                 </row>
1041                 <row>
1042                   <entry>
1043                     <para> <literal>ENOTSUPP</literal></para>
1044                   </entry>
1045                   <entry>
1046                     <para>Advice type is not supported.</para>
1047                   </entry>
1048                 </row>
1049               </tbody>
1050             </tgroup>
1051           </informaltable>
1052           </para>
1053       </section>
1054   </section>
1055   <section xml:id="dbdoclet.50438215_marker-1297700">
1056     <title>Example Using the <literal>llapi</literal> Library</title>
1057     <para>Use <literal>llapi_file_create</literal> to set Lustre software properties for a new file.
1058       For a synopsis and description of <literal>llapi_file_create</literal> and examples of how to
1059       use it, see <xref linkend="configurationfilesmoduleparameters"/>.</para>
1060     <para>You can set striping from inside programs like <literal>ioctl</literal>. To compile the sample program, you need to install the Lustre client source RPM.</para>
1061     <para><emphasis role="bold">A simple C program to demonstrate striping API - libtest.c</emphasis></para>
1062     <programlisting>
1063 /* -*- mode: c; c-basic-offset: 8; indent-tabs-mode: nil; -*-
1064  * vim:expandtab:shiftwidth=8:tabstop=8:
1065  *
1066  * lustredemo - a simple example of lustreapi functions
1067  */
1068 #include &lt;stdio.h&gt;
1069 #include &lt;fcntl.h&gt;
1070 #include &lt;dirent.h&gt;
1071 #include &lt;errno.h&gt;
1072 #include &lt;stdlib.h&gt;
1073 #include &lt;lustre/lustreapi.h&gt;
1074 #define MAX_OSTS 1024
1075 #define LOV_EA_SIZE(lum, num) (sizeof(*lum) + num * sizeof(*lum-&gt;lmm_objects))
1076 #define LOV_EA_MAX(lum) LOV_EA_SIZE(lum, MAX_OSTS)
1077
1078 /*
1079  * This program provides crude examples of using the lustreapi API functions
1080  */
1081 /* Change these definitions to suit */
1082
1083 #define TESTDIR &quot;/tmp&quot;           /* Results directory */
1084 #define TESTFILE &quot;lustre_dummy&quot;  /* Name for the file we create/destroy */
1085 #define FILESIZE 262144                    /* Size of the file in words */
1086 #define DUMWORD &quot;DEADBEEF&quot;       /* Dummy word used to fill files */
1087 #define MY_STRIPE_WIDTH 2                  /* Set this to the number of OST required */
1088 #define MY_LUSTRE_DIR &quot;/mnt/lustre/ftest&quot;
1089
1090 int close_file(int fd)
1091 {
1092         if (close(fd) &lt; 0) {
1093                 fprintf(stderr, &quot;File close failed: %d (%s)\n&quot;, errno, strerror(errno));
1094                 return -1;
1095         }
1096         return 0;
1097 }
1098
1099 int write_file(int fd)
1100 {
1101         char *stng =  DUMWORD;
1102         int cnt = 0;
1103
1104         for( cnt = 0; cnt &lt; FILESIZE; cnt++) {
1105                 write(fd, stng, sizeof(stng));
1106         }
1107         return 0;
1108 }
1109 /* Open a file, set a specific stripe count, size and starting OST
1110  *    Adjust the parameters to suit */
1111 int open_stripe_file()
1112 {
1113         char *tfile = TESTFILE;
1114         int stripe_size = 65536;    /* System default is 4M */
1115         int stripe_offset = -1;     /* Start at default */
1116         int stripe_count = MY_STRIPE_WIDTH;  /*Single stripe for this demo*/
1117         int stripe_pattern = 0;     /* only RAID 0 at this time */
1118         int rc, fd;
1119
1120         rc = llapi_file_create(tfile,
1121                         stripe_size,stripe_offset,stripe_count,stripe_pattern);
1122         /* result code is inverted, we may return -EINVAL or an ioctl error.
1123          * We borrow an error message from sanity.c
1124          */
1125         if (rc) {
1126                 fprintf(stderr,&quot;llapi_file_create failed: %d (%s) \n&quot;, rc, strerror(-rc));
1127                 return -1;
1128         }
1129         /* llapi_file_create closes the file descriptor, we must re-open */
1130         fd = open(tfile, O_CREAT | O_RDWR | O_LOV_DELAY_CREATE, 0644);
1131         if (fd &lt; 0) {
1132                 fprintf(stderr, &quot;Can't open %s file: %d (%s)\n&quot;, tfile, errno, strerror(errno));
1133                 return -1;
1134         }
1135         return fd;
1136 }
1137
1138 /* output a list of uuids for this file */
1139 int get_my_uuids(int fd)
1140 {
1141         struct obd_uuid uuids[1024], *uuidp;        /* Output var */
1142         int obdcount = 1024;
1143         int rc,i;
1144
1145         rc = llapi_lov_get_uuids(fd, uuids, &amp;obdcount);
1146         if (rc != 0) {
1147                 fprintf(stderr, &quot;get uuids failed: %d (%s)\n&quot;,errno, strerror(errno));
1148         }
1149         printf(&quot;This file system has %d obds\n&quot;, obdcount);
1150         for (i = 0, uuidp = uuids; i &lt; obdcount; i++, uuidp++) {
1151                 printf(&quot;UUID %d is %s\n&quot;,i, uuidp-&gt;uuid);
1152         }
1153         return 0;
1154 }
1155
1156 /* Print out some LOV attributes. List our objects */
1157 int get_file_info(char *path)
1158 {
1159
1160         struct lov_user_md *lump;
1161         int rc;
1162         int i;
1163
1164         lump = malloc(LOV_EA_MAX(lump));
1165         if (lump == NULL) {
1166                 return -1;
1167         }
1168
1169         rc = llapi_file_get_stripe(path, lump);
1170
1171         if (rc != 0) {
1172                 fprintf(stderr, &quot;get_stripe failed: %d (%s)\n&quot;,errno, strerror(errno));
1173                 return -1;
1174         }
1175
1176         printf(&quot;Lov magic %u\n&quot;, lump-&gt;lmm_magic);
1177         printf(&quot;Lov pattern %u\n&quot;, lump-&gt;lmm_pattern);
1178         printf(&quot;Lov object id %llu\n&quot;, lump-&gt;lmm_object_id);
1179         printf(&quot;Lov stripe size %u\n&quot;, lump-&gt;lmm_stripe_size);
1180         printf(&quot;Lov stripe count %hu\n&quot;, lump-&gt;lmm_stripe_count);
1181         printf(&quot;Lov stripe offset %u\n&quot;, lump-&gt;lmm_stripe_offset);
1182         for (i = 0; i &lt; lump-&gt;lmm_stripe_count; i++) {
1183                 printf(&quot;Object index %d Objid %llu\n&quot;, lump-&gt;lmm_objects[i].l_ost_idx, lump-&gt;lmm_objects[i].l_object_id);
1184         }
1185
1186         free(lump);
1187         return rc;
1188
1189 }
1190
1191 /* Ping all OSTs that belong to this filesystem */
1192 int ping_osts()
1193 {
1194         DIR *dir;
1195         struct dirent *d;
1196         char osc_dir[100];
1197         int rc;
1198
1199         sprintf(osc_dir, &quot;/proc/fs/lustre/osc&quot;);
1200         dir = opendir(osc_dir);
1201         if (dir == NULL) {
1202                 printf(&quot;Can't open dir\n&quot;);
1203                 return -1;
1204         }
1205         while((d = readdir(dir)) != NULL) {
1206                 if ( d-&gt;d_type == DT_DIR ) {
1207                         if (! strncmp(d-&gt;d_name, &quot;OSC&quot;, 3)) {
1208                                 printf(&quot;Pinging OSC %s &quot;, d-&gt;d_name);
1209                                 rc = llapi_ping(&quot;osc&quot;, d-&gt;d_name);
1210                                 if (rc) {
1211                                         printf(&quot;  bad\n&quot;);
1212                                 } else {
1213                                         printf(&quot;  good\n&quot;);
1214                                 }
1215                         }
1216                 }
1217         }
1218         return 0;
1219
1220 }
1221
1222 int main()
1223 {
1224         int file;
1225         int rc;
1226         char filename[100];
1227         char sys_cmd[100];
1228
1229         sprintf(filename, &quot;%s/%s&quot;,MY_LUSTRE_DIR, TESTFILE);
1230
1231         printf(&quot;Open a file with striping\n&quot;);
1232         file = open_stripe_file();
1233         if ( file &lt; 0 ) {
1234                 printf(&quot;Exiting\n&quot;);
1235                 exit(1);
1236         }
1237         printf(&quot;Getting uuid list\n&quot;);
1238         rc = get_my_uuids(file);
1239         printf(&quot;Write to the file\n&quot;);
1240         rc = write_file(file);
1241         rc = close_file(file);
1242         printf(&quot;Listing LOV data\n&quot;);
1243         rc = get_file_info(filename);
1244         printf(&quot;Ping our OSTs\n&quot;);
1245         rc = ping_osts();
1246
1247         /* the results should match lfs getstripe */
1248         printf(&quot;Confirming our results with lfs getstripe\n&quot;);
1249         sprintf(sys_cmd, &quot;/usr/bin/lfs getstripe %s/%s&quot;, MY_LUSTRE_DIR, TESTFILE);
1250         system(sys_cmd);
1251
1252         printf(&quot;All done\n&quot;);
1253         exit(rc);
1254 }
1255 </programlisting>
1256     <para><emphasis role="bold">Makefile for sample application:</emphasis></para>
1257     <screen> 
1258 gcc -g -O2 -Wall -o lustredemo libtest.c -llustreapi
1259 clean:
1260 rm -f core lustredemo *.o
1261 run: 
1262 make
1263 rm -f /mnt/lustre/ftest/lustredemo
1264 rm -f /mnt/lustre/ftest/lustre_dummy
1265 cp lustredemo /mnt/lustre/ftest/
1266 </screen>
1267     <section remap="h5">
1268       <title>See Also</title>
1269       <itemizedlist>
1270         <listitem>
1271           <para>
1272             <xref linkend="dbdoclet.50438215_30970"/>
1273     </para>
1274         </listitem>
1275         <listitem>
1276           <para>
1277             <xref linkend="dbdoclet.50438215_50149"/>
1278     </para>
1279         </listitem>
1280         <listitem>
1281           <para>
1282             <xref linkend="dbdoclet.50438215_86607"/>
1283     </para>
1284         </listitem>
1285         <listitem>
1286           <para>
1287             <xref linkend="dbdoclet.50438215_12433"/>
1288     </para>
1289         </listitem>
1290       </itemizedlist>
1291     </section>
1292   </section>
1293 </chapter>