--- /dev/null
+#LyX 1.3 created this file. For more info see http://www.lyx.org/
+\lyxformat 221
+\textclass article
+\begin_preamble
+\usepackage{listings}
+\usepackage{color}
+\usepackage{courier}
+
+%\usepackage{pt}
+%\usepackage{units}
+%\usepackage{coz}
+
+%\usepackage{epsf}
+%\usepackage{lncsexample}
+%\conttrue
+%\usepackage{times}
+%\usepackage{top}
+%\usepackage{graphicx}
+\newcommand{\lst}[3] {
+ \noindent\rule[-0.3mm]{\textwidth}{0.3mm}\vspace{-0.3mm}
+ \lstinputlisting[caption={#2},
+ label={#1},
+ showstringspaces=false,
+ numbers=left,
+ stepnumber=1,
+ frame=bottomline,
+ language=C,
+ extendedchars=true,
+ basicstyle=\small\tt,
+ numberstyle=\tiny,
+ keywordstyle=\color{red},
+ commentstyle=\color{cyan},
+ stringstyle=\color{green},
+ directivestyle=\color{magenta},
+ emph={1, 2, 3, 4, 5, 6, 7, 8, 9, 0, NULL, lustre, CFS},
+ emphstyle=\color{blue},
+ breaklines=true,
+ #3]
+ {#1}\vspace{0.3mm}
+}
+\end_preamble
+\language english
+\inputencoding auto
+\fontscheme times
+\graphics default
+\paperfontsize default
+\spacing single
+\papersize Default
+\paperpackage a4
+\use_geometry 1
+\use_amsmath 0
+\use_natbib 0
+\use_numerical_citations 0
+\paperorientation portrait
+\leftmargin 1in
+\topmargin 0.5in
+\rightmargin 0.5in
+\bottommargin 0.5in
+\secnumdepth 4
+\tocdepth 4
+\paragraph_separation skip
+\defskip medskip
+\quotes_language english
+\quotes_times 2
+\papercolumns 1
+\papersides 1
+\paperpagestyle default
+
+\layout Title
+
+
+\noun on
+Storage Management File System
+\layout Date
+
+14 February 2005
+\layout Author
+
+Peter Braam, Mike Pershin
+\layout Standard
+
+
+\begin_inset LatexCommand \tableofcontents{}
+
+\end_inset
+
+
+\layout Section
+
+Requirements
+\layout Standard
+
+In this document SMFS (Storage Management File System) is described.
+ It is required to do the following:
+\layout Itemize
+
+works as transparent pseudo-filesystem on the top of real filesystem
+\layout Itemize
+
+have access to Lustre's API like llog and btree
+\layout Itemize
+
+be portable to other file systems than ext3
+\layout Itemize
+
+be able to call side-functions from other modules and plugins using own
+ API.
+\layout Standard
+
+All features can exist together and should not affect bottom filesystem
+ in any way.
+\layout Section
+
+Architectural overview
+\layout Standard
+
+In order to achieve requirements SMFS is expected to have the following
+ major functionality divisions:
+\layout Description
+
+Backstore\SpecialChar ~
+FS - main module that is placed in Lustre stack on top of backing
+ FS.
+ It is pseudo filesystem that just pass all operations to real one and can
+ invoke side-functions for each filesystem operation.
+ There is plugin API that can be used by special modules (plugins) to register
+ list of functions for various filesystem operations.
+\layout Description
+
+Plugins - major reason for SMFS existence is possibility to call side-functions
+ for any filesystem operation.
+ Let's call module with such functionality as SMFS plugin.
+ Each plugin is initialized via options when SMFS is mounted and corresponds
+ to certain method tables.
+ Plugins should register in SMFS using plugins API after that SMFS will
+ call their functions for specified filesystem operations.
+
+\layout Description
+
+Upcalls - very similar with plugins but more generic.
+ Modules can use this API to place own hooks in SMFS.
+ It can be needed for notification or taking control from SMFS by upper-level.
+
+\layout Section
+\pagebreak_top
+High Level Design
+\layout Subsection
+
+
+\noun on
+Functional Specification
+\layout Subsubsection
+
+Backstore FS operations
+\layout Standard
+
+SMFS creates copy of backstore FS objects when needed but setup own super/inode/
+file operations and save pointer to original one.
+ SMFS should not affect the Lustre IO path.
+ Backstore FS should remains operational in case of SMFS error as long as
+ it possible.
+\layout Subsubsection
+
+SMFS plugins/upcalls managment
+\layout Standard
+
+SMFS can registers/deregisters plugin.
+ Plugins are activated after SMFS initialization and can register own functions
+ in SMFS using special API.
+
+\layout Standard
+
+All plugins are linked in the list and their functions are invoked for several
+ filesystem operations if defined.
+
+\layout Standard
+
+Deactivated process starts when SMFS uninitialization occur or ioctl is
+ received from user.
+ Ioctl interface or procfs (sysfs later) can be used also for plugin management.
+\layout Standard
+
+Upcall are similar to plugins but more generic.
+ There is upcall API in SMFS.
+ Upcalls are also organized in linked list and called one by one where needed.
+
+\layout Subsubsection
+
+SMFS plugin API
+\layout Standard
+
+Plugins should be able to register and deregister with SMFS.
+ For these purposes SMFS has two methods
+\layout Standard
+
+
+\noun on
+Prototype
+\noun default
+:
+\layout LyX-Code
+
+int smfs_register_plugin(struct smfs_plugin);
+\layout LyX-Code
+
+int smfs_deregister_plugin(int type);
+\layout Standard
+
+Before calling this function plugin should fill struct smfs_plugin with
+ own data.
+ Registration process will check it and add to the list if possible.
+ Deregistration takes type of plugin, search it in list and delete it.
+\layout Standard
+
+Plugins can insert own function hooks into SMFS.
+ Two major functions that can be setted by plugins are pre- and post-hook.
+ They will be called before filesystem operation and after it.
+\layout Standard
+
+
+\noun on
+Prototype:
+\layout LyX-Code
+
+/* this is prototype for plugin hook */
+\layout LyX-Code
+
+int smfs_hook_func (int hook_opcode, void * parameters);
+\layout Standard
+
+Here we pass to plugin hook_opcode and several parameters.
+ For each opcode paramaters can be different.
+\layout Standard
+
+
+\emph on
+Hook_opcode
+\emph default
+ is a number from enumerated list of all possible operations where hooks
+ are used.
+\layout Subsubsection
+
+SMFS upcall API
+\layout Standard
+
+Plugin API is defined for using with filesystem operations specially.
+ For cases where more generic functionality is needed upcall API can be
+ used.
+
+\layout Standard
+
+
+\noun on
+Prototype:
+\layout LyX-Code
+
+int smfs_upcall (int type, void * arg);
+\layout Standard
+
+Here
+\emph on
+type
+\emph default
+ is number from enumerated list of upcalls,
+\emph on
+arg
+\emph default
+ is parameter or structure with parameters related to this upcall.
+ Upcalls can be used for notification purposes.
+ Upcalls are also widely used with plugins to get some data from they or
+ do some action.
+\layout Standard
+
+Due to upcall origin there can be no some modules while SMFS already is
+ loaded.
+ Therefore it is possible that not all functionality presents at the moment.
+ Upcall API should notify caller about this.
+\layout Subsection
+
+
+\noun on
+Use cases
+\layout Standard
+
+There are several plugins that use plugin API.
+ They should fit in proposed API well.
+\layout Subsubsection
+
+KML
+\layout Standard
+
+KML module designed to support writeback caching.
+ KML is quite simple plugin that receive notification about FS operations
+ and write they to the log using llog API.
+\layout Standard
+
+KML use post_op function for logging, that are called after FS operations.
+ It should have also pre_ops for transaction handling.
+ The SMFS functionality is sufficient for it.
+
+\layout Subsubsection
+
+LRU
+\layout Standard
+
+LRU is designed for cache management in local MDS on client, but can be
+ used also for HSM in future.
+ LRU module defines both pre_op and post_op.
+ Pre_op is quit simple and is used mostly to check do we need purge cache
+ or not.
+ Post_op does cache operations.
+ There is special thread for purging cache, it can be awaken from pre_op
+ but.
+ SMFS plugin API functionality is sufficient for this module.
+\layout Subsubsection
+
+COW
+\layout Standard
+
+This module supports snapfs working over SMFS.
+ This module use all functionality of SMFS plugin API - pre_op and post_op.
+ Meanwhile it doesn't need extra functionality and fits in existent API.
+\layout Subsection
+
+
+\noun on
+Logic Specification
+\layout Subsubsection
+
+Backstore FS operations
+\layout Standard
+
+For succefull functioning SMFS should maintain copies of many filesystem
+ objects such as superblock, inodes, files, etc.
+ and replace their operations with own.
+ Therefore each filesystem object in SMFS has its origin object in backstore
+ FS and there is pointer on it.
+ While operating SMFS should maintain own object in consistency with real
+ objects.
+\layout Standard
+
+Example of superblock operation handling:
+\layout LyX-Code
+
+smfs_dirty/write_inode(struct inode * inode)
+\layout LyX-Code
+
+{
+\layout LyX-Code
+
+ /* getting saved backstore fs inode */
+\layout LyX-Code
+
+ backfs_inode = I2CI(inode);
+\layout LyX-Code
+
+ /* here can be some pre-work */
+\layout LyX-Code
+
+ ...
+\layout LyX-Code
+
+ /* call real operation */
+\layout LyX-Code
+
+ backfs_sb->s_op->dirty/write_inode(backfs_inode);
+\layout LyX-Code
+
+
+\layout LyX-Code
+
+ /* postprocessing */
+\layout LyX-Code
+
+ duplicate_inode(inode, backfs_inode);
+\layout LyX-Code
+
+}
+\layout Subsubsection
+
+Fsfilt operations handlilng
+\layout Standard
+
+Fsfilt operations are handled like ordinary FS operations.
+ SMFS store fsfilt operations that corresponds to backstore FS, fsfilt module
+ just use operations declared for SMFS.
+ SMFS calls real operation with some framework.
+ There are several common steps to pass fsfilt operations to real FS:
+\layout Itemize
+
+get backfs sb and inode from SMFS struct inode
+\layout Itemize
+
+get backfs fsfilt_operations from superblock
+\layout Itemize
+
+call needed operation using backfs objects
+\layout Standard
+
+Transactions in SMFS are handled using fsfilter operation for backstore
+ FS.
+ SMFS make fs_start and fs_commit in all cases where it is needed.
+ Therefore these all extra changes will be included into current transaction:
+\layout LyX-Code
+
+smfs_trans_start()
+\layout LyX-Code
+
+...
+\layout LyX-Code
+
+SMFS_HOOK()
+\layout LyX-Code
+
+...
+\layout LyX-Code
+
+smfs_trans_commit()
+\layout Standard
+
+Transaction handler are passed to plugin in hook and can be used by plugin
+ to do proper transaction handling when it is needed.
+\layout Subsubsection
+
+SMFS plugin activation/deactivation
+\layout Standard
+
+Now all plugins are compiled in SMFS module and are initialized via options
+ when SMFS is mounting.
+
+\layout Standard
+
+Options should be passed as mountfsoptions like this:
+\emph on
+...
+ [kml|cache|snap...]
+\layout Standard
+
+SMFS parses module options and call initialization method for selected plugins.
+ It is possible to activate/deactivate plugins using ioctl (procfs or sysfs)
+ later.
+
+\layout Standard
+
+While initialization plugins should registers with SMFS:
+\layout LyX-Code
+
+smfs_register_plugin(parameters);
+\layout Standard
+
+Parameters are struct smfs_plugin filled with valid data:
+\layout Itemize
+
+type of plugin - to distinguish it from anothers,
+\layout Itemize
+
+pre_op function - function that will be called before fs operation,
+\layout Itemize
+
+post_op function - function that will be called after fs operation,
+\layout Itemize
+
+helper function - helper function.
+ See Plugin API,
+\layout Itemize
+
+any private data - data which will be return to plugin in each call.
+\layout Subsubsection
+
+SMFS hook invokation
+\layout Standard
+
+SMFS place special wrapper in own filesystem operations to call plugins
+ hook:
+\layout LyX-Code
+
+/* wrapper that calls all hooks walking through the list */
+\layout LyX-Code
+
+#define SMFS_HOOK(opcode,...)
+\backslash
+
+\layout LyX-Code
+
+do {
+\backslash
+
+\layout LyX-Code
+
+ smfs_hooks_op * hops;
+\backslash
+
+\layout LyX-Code
+
+ ...
+
+\backslash
+
+\layout LyX-Code
+
+ list_for_each_entry(plugin_list, hops, smh_list) {
+\backslash
+
+\layout LyX-Code
+
+ ...
+
+\backslash
+
+\layout LyX-Code
+
+ rc = hops->smh_hook_op(opcode, ...);
+\backslash
+
+\layout LyX-Code
+
+ }
+\backslash
+
+\layout LyX-Code
+
+ ...
+
+\backslash
+
+\layout LyX-Code
+
+} while(0)
+\layout Standard
+
+Hooks are placed in SMFS methods before and after calling backstore FS operation
+s:
+\layout LyX-Code
+
+/* this is how to SMFS uses hooks */
+\layout LyX-Code
+
+smfs_some_op()
+\layout LyX-Code
+
+{
+\layout LyX-Code
+
+ struct inode * backfs_inode = I2CI(inode);
+\layout LyX-Code
+
+ struct smfs_file_info *sfi;
+\layout LyX-Code
+
+
+\layout LyX-Code
+
+ SMFS_HOOK(..., hook_opcode , ..., PRE_HOOK, ...);
+\layout LyX-Code
+
+ backfs_inode->i_fop->some_op(sfi->c_file, ...);
+\layout LyX-Code
+
+ SMFS_HOOK(..., hook_opcode, ..., POST_HOOK, ...);
+\layout LyX-Code
+
+}
+\layout Subsubsection
+
+SMFS upcall usage
+\layout Standard
+
+Upcalls can be placed in any place in SMFS, where it is needed.
+\layout LyX-Code
+
+/* example of upcalls handling */
+\layout LyX-Code
+
+int smfs_upcall(int type, void * arg)
+\layout LyX-Code
+
+{
+\layout LyX-Code
+
+ list_for_each(upcall_list)
+\layout LyX-Code
+
+ entry->upcall(type, arg);
+\layout LyX-Code
+
+}
+\layout LyX-Code
+
+\layout LyX-Code
+
+/* example of using upcall */
+\layout LyX-Code
+
+int check_uninitialization()
+\layout LyX-Code
+
+{
+\layout LyX-Code
+
+ ...
+\layout LyX-Code
+
+ smfs_upcall(SMFS_UP_SMTH, ¶meter);
+\layout LyX-Code
+
+ ...
+\layout LyX-Code
+
+}
+\layout Section
+\pagebreak_top
+Plugins API
+\layout Subsection
+
+
+\noun on
+Functional Specification
+\layout Standard
+
+Each plugin has own unique name and corresponding type.
+ There is enumerated list of plugins types.
+ Any new plugin should use next sequence number as type and name which not
+ used yet by other plugins.
+ SMFS exports two functions for plugins:
+\emph on
+smfs_register_plugin()
+\emph default
+ and
+\emph on
+smfs_deregister_plugin().
+
+\emph default
+
+\layout Standard
+
+SMFS defines format of function that can be used by plugins to invoke own
+ methods.
+ Plugins should pass pointers on desired function while registration.
+\layout Standard
+
+After successfull registration plugin's function will be called when selected
+ FS operations are occur.
+ SMFS passes to plugin inode structre, dentry, hook opcode and fsfilt transactio
+n handle if exists.
+
+\layout Standard
+
+Plugin should return zero value in case of successfull completion and error
+ code otherwise.
+ Plugin should decide how to handle error by itself.
+ In case of critical error it can even stops operating or deregisters itself
+ from SMFS.
+ If plugin have no function for selected
+\emph on
+hook_opcode
+\emph default
+ it has to return also
+\emph on
+0.
+\layout Subsection
+
+
+\noun on
+Logic Specificaion
+\layout Paragraph*
+
+Plugin identification
+\layout Standard
+
+There are defined types of plugins:
+\layout LyX-Code
+
+#define SMFS_PLG_KML 0x01
+\layout LyX-Code
+
+#define SMFS_PLG_LRU 0x02
+\layout LyX-Code
+
+...
+\layout LyX-Code
+
+\layout Paragraph*
+
+Plugin data
+\layout Standard
+
+Main structure for communication between plugin and SMFS.
+
+\layout LyX-Code
+
+struct smfs_plugin {
+\layout LyX-Code
+
+ struct list_head plist;
+\layout LyX-Code
+
+ int type;
+\layout LyX-Code
+
+ smfs_hook_func * pre_op;
+\layout LyX-Code
+
+ smfs_hook_func * post_op;
+\layout LyX-Code
+
+ smfs_helper * helper;
+\layout LyX-Code
+
+ void * plugin_data;
+\layout LyX-Code
+
+}
+\layout Standard
+
+Where
+\layout Itemize
+
+
+\emph on
+type
+\emph default
+ - plugins type,
+\layout Itemize
+
+
+\emph on
+pre_hook_func
+\emph default
+ - function to handle pre-operations,
+\layout Itemize
+
+
+\emph on
+post_hook_func
+\emph default
+ - function to handle post-hook operation,
+\layout Itemize
+
+helper - call to various helper functions needed by SMFS,
+\layout Itemize
+
+plugin_data - private plugin data.
+\layout Paragraph*
+
+Plugin registration
+\layout Standard
+
+SMFS provide following method for registration:
+\layout LyX-Code
+
+int smfs_register_plugin (struct smfs_plugin *);
+\layout Standard
+
+SMFS will return 0 in case of successfull registration or error code otherwise.
+
+\layout Paragraph*
+
+Plugins operation
+\layout Standard
+
+Plugin's hook function must have the same type:
+\layout LyX-Code
+
+typedef int smfs_hook_func (int opcode, void * parameter);
+\layout Standard
+
+Parameter can contains:
+\layout Itemize
+
+inode - SMFS inode
+\layout Itemize
+
+dentry - SMFS dentry
+\layout Itemize
+
+new_inode - used in rename
+\layout Itemize
+
+new_dentry - used in rename
+\layout Itemize
+
+opcode - hook code
+\layout Itemize
+
+handle - fsfilt transaction if exists
+\layout Standard
+
+There are opcodes for now:
+\layout LyX-Code
+
+#define HOOK_CREATE 1
+\layout LyX-Code
+
+#define HOOK_LOOKUP 2
+\layout LyX-Code
+
+#define HOOK_LINK 3
+\layout LyX-Code
+
+#define HOOK_UNLINK 4
+\layout LyX-Code
+
+#define HOOK_SYMLINK 5
+\layout LyX-Code
+
+#define HOOK_MKDIR 6
+\layout LyX-Code
+
+#define HOOK_RMDIR 7
+\layout LyX-Code
+
+#define HOOK_MKNOD 8
+\layout LyX-Code
+
+#define HOOK_RENAME 9
+\layout LyX-Code
+
+#define HOOK_SETATTR 10
+\layout LyX-Code
+
+#define HOOK_WRITE 11
+\layout LyX-Code
+
+#define HOOK_READDIR 12
+\layout LyX-Code
+
+#define HOOK_MAX 12
+\layout Standard
+
+Plugin may use opcode as index for array of some type.
+ It can be table of functions for each hook type for example:
+\layout LyX-Code
+
+static cache_hook_op cache_space_hook_ops[HOOK_MAX + 1] = {
+\layout LyX-Code
+
+ [HOOK_CREATE] cache_space_hook_create,
+\layout LyX-Code
+
+ [HOOK_LOOKUP] cache_space_hook_lookup,
+\layout LyX-Code
+
+ [HOOK_LINK] cache_space_hook_link,
+\layout LyX-Code
+
+ [HOOK_UNLINK] cache_space_hook_unlink,
+\layout LyX-Code
+
+ [HOOK_SYMLINK] cache_space_hook_create,
+\layout LyX-Code
+
+ [HOOK_MKDIR] cache_space_hook_mkdir,
+\layout LyX-Code
+
+ [HOOK_RMDIR] cache_space_hook_rmdir,
+\layout LyX-Code
+
+ [HOOK_MKNOD] cache_space_hook_create,
+\layout LyX-Code
+
+ [HOOK_RENAME] cache_space_hook_rename,
+\layout LyX-Code
+
+ [HOOK_SETATTR] NULL,
+\layout LyX-Code
+
+ [HOOK_WRITE] NULL,
+\layout LyX-Code
+
+ [HOOK_READDIR] NULL,
+\layout LyX-Code
+
+};
+\layout Subsubsection*
+
+Helpers
+\layout Standard
+
+Plugins can register helper function.
+ There are several places where helpers are needed:
+\layout Itemize
+
+smfs uninitialization - after deleting plugin from list it can be usefull
+ to invoke plugin deactivation
+\layout Itemize
+
+init_inode_info - several plugins can need to initialize own info on per-inode
+ basis
+\layout Itemize
+
+calculate size for transaction - if plugin will participate into transaction
+ it should return extra size value.
+\layout Itemize
+
+etc.
+\layout LyX-Code
+
+smfs_helper(int code, void * parameter);
+\layout Standard
+
+Where
+\emph on
+code
+\emph default
+ is type of helper:
+\layout LyX-Code
+
+#define SMFS_HLP_EXIT 0x01
+\layout LyX-Code
+
+#define SMFS_HLP_INODE 0x02
+\layout LyX-Code
+
+#define SMFS_HLP_TSIZE 0x03
+\layout Subsection
+
+
+\noun on
+Use cases
+\layout Standard
+
+SMFS returns zero value in case of successfull registration/deregistration
+ or error code otherwise.
+\layout Standard
+
+Plugins will return zero value in case of successfull execution hook function
+ or error code otherwise.
+\layout Paragraph*
+
+New plugin design example
+\layout LyX-Code
+
+//add new type
+\layout LyX-Code
+
+#define SMFS_PLG_NEW 0x03
+\layout LyX-Code
+
+\layout LyX-Code
+
+//prepare hook functions
+\layout LyX-Code
+
+int new_plugin_pre_function(int opcode, void * parameter)
+\layout LyX-Code
+
+{
+\layout LyX-Code
+
+ //let it be just statistic collector
+\layout LyX-Code
+
+ array[opcode]++;
+\layout LyX-Code
+
+}
+\layout LyX-Code
+
+\layout LyX-Code
+
+int new_plugin_init ()
+\layout LyX-Code
+
+{
+\layout LyX-Code
+
+ struct smfs_plugin data;
+\layout LyX-Code
+
+ //prepare data
+\layout LyX-Code
+
+ data.type = SMFS_PLG_NEW;
+\layout LyX-Code
+
+ data.pre_op = &new_plugin_pre_function;
+\layout LyX-Code
+
+ data.post_op = NULL;
+\layout LyX-Code
+
+ //regisration here
+\layout LyX-Code
+
+ err = smfs_register_plugin(&data);
+\layout LyX-Code
+
+\layout LyX-Code
+
+}
+\layout Section
+\pagebreak_top
+Upcall API
+\layout Standard
+
+SMFS provides the ability to place a call to other modules.
+ This can be needed when some notification needed from SMFS for example.
+\layout Standard
+
+There are quite a few things to keep in mind:
+\layout Enumerate
+
+The module that causes the mount of smfs, like the low level OSS and MDS
+ server will want to register methods for upcalls.
+\layout Enumerate
+
+Other modules that are not mounting need to be able to find smfs and register
+ upcall methods.
+\layout Enumerate
+
+When these registration of upcall methods have not happened, we should get
+ reasonable error handling.
+ Oopses and hangs are not permitted.
+\layout Subsection
+
+
+\noun on
+Functional Specification
+\layout Standard
+
+We can use mechanism similar to plugin API but it is more generic.
+ Upcall is invoked in particular place or when some conditions are occure.
+ So each upcall has own type.
+ There can be several upcalls with one type simultaneously.
+ Therefore we can introduce set of types for various upcalls and make generic
+ call function like
+\emph on
+smfs_upcall(int type, void * arg)
+\emph default
+.
+ Module should register upcall handler in SMFS with help of registering
+ function.
+ SMFS has list of upcalls and will call functions for each upcall type walking
+ through list.
+
+\layout Subsection
+
+
+\noun on
+Logic Specification
+\layout Standard
+
+There are defined types of upcalls:
+\layout LyX-Code
+
+#define SMFS_UP_SMTH 0x01
+\layout LyX-Code
+
+#define SMFS_UP_...
+ 0x02
+\layout LyX-Code
+
+...
+\layout LyX-Code
+
+\layout Standard
+
+SMFS provide following method for registration/deregistration:
+\layout LyX-Code
+
+int smfs_register_upcall (struct smfs_upcall *);
+\layout LyX-Code
+
+int smfs_deregister_upcall (struct smfs_upcall *);
+\layout Standard
+
+SMFS will return 0 in case of successfull registration or error code otherwise.
+\layout Standard
+
+SMFS will invoke function for each upcall:
+\layout LyX-Code
+
+typedef int smfs_upcall_func (int upcall_type, void * arg);
+\layout Standard
+
+Here:
+\layout Itemize
+
+upcall_type - upcall type.
+ It is define where upcall was invoked and under what conditions;
+\layout Itemize
+
+arg - any desired arguments.
+\layout Subsection
+
+
+\noun on
+Use cases
+\layout Standard
+
+SMFS returns zero value in case of successfull registration/deregistration
+ or error code otherwise.
+\layout Standard
+
+Modules should be aware about conditions under what upcall is called - locking,
+ for example.
+
+\the_end