What is ficl?

Ficl is a lightweight, embeddable scripting language designed to be incorporated into other programs, including memory constrained embedded systems. Ficl conforms to the 1994 ANSI Standard for Forth, and provides several useful extensions including OOP that can wrap compiled code and hardware interfaces. 

Unlike Lua or Python, Ficl acts as a component of your system: you feed it stuff to do, it does the stuff, and comes back to you for more. You can export compiled code to Ficl, execute Ficl code from your compiled code, or interact with a read-execute-print loop. Your choice. Ficl includes a simple but capable object model that can wrap existing data structures. 

Ficl vs. other Interpreters

Ficl Design goals

• Scripting, prototyping, and extension language for systems written also in C/C++
• Target 32 & 64 bit processors, including embedded systems with limited memory
• Wrap functions and data structures written in C/C++
• Conform to the Forth DPANS 94
• Minimize porting effort - require an ANSI C runtime environment and minimal glue code
• Provide Object Oriented extensions that can wrap system functions and structures

References: More information on Ficl

• Web home of Ficl
• Github Repository
• Try Ficl in your browser
• OO in C background notes
• Manuscript of Ficl article for January 1999 Dr. Dobb's Journal
• 1998 FORML Conference paper - OO Programming in Ficl

Forth and Threaded Interpretive Languages

• An Introduction to Forth using Stack Flow (start here if you're new to Forth)
• Phil Burk's Forth Tutorial
• An excellent Forth Primer by Hans Bezemer
• Anton Ertl's description of Threaded Code
• American National Standard for Forth
• Forth literature index on Taygeta
• Tutorials from Forth Interest Group

LICENSE and DISCLAIMER

Copyright (c) 1997-2026 John Sadler, All rights reserved.

I am interested in hearing from anyone who uses ficl. If you have a problem, a success story, a defect, an enhancement request, or if you would like to contribute to a ficl release, please contact me on Sourceforge. 

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
   may be used to endorse or promote products derived from this software
   without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS''
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.

Getting Started: Porting Ficl to your system

Edit the definitions (in sysdep.c) of ficlMalloc, ficlFree, ficlRealloc, and ficlTextOut for your application and target. Use testmain.c as a guide to installing the ficl system and one or more virtual machines into your code. 

Ficl includes portable double precision math routines that work for 32 and 64 bit machines. You may replace them with machine-dependent versions by #defining PORTABLE_LONGMULDIV to 0 in sysdep.h and implementing your versions of ficlLongMul and ficlLongDiv in sysdep.c.  

Build controls

Softwords

Application Programming Interface

FICL_SYSTEM *ficlInitSystemEx(FICL_SYSTEM_INFO *fsi)

Binds a global dictionary to the interpreter system and initializes the dict to contain the ANSI CORE wordset. You can specify the dictionary size, text output function, and an extension pointer via the FICL_SYSTEM_INFO structure. After initialization, ficl manages the dictionary. The dictionary needs to be at least large enough to hold the precompiled part. Try 5K cells to start. Use .dictto find out how much of the dictionary is used at any time.

FICL_SYSTEM *ficlInitSystem(int nDictCells)

Deprecated. Initializes Ficl's shared system data structures, and creates the dictionary allocating the specified number of CELLs from the heap (by a call to ficlMalloc). Use ficlInitSystemEx for new code.

void ficlTermSystem(FICL_SYSTEM *pSys)

Deletes the system dictionary and all virtual machines that were created with ficlNewVM. Call this function to reclaim all memory used by the dictionary and VMs. Any uses of the memory allocation words (allocate and resize) are your problem.

int ficlEvaluate(FICL_VM *pVM, const char *pText)

Evaluates a block of input text in the context of the specified interpreter. Also sets SOURCE-ID properly. Use this function when passing a hard-coded string to the Ficl interpreter.

int ficlExec(FICL_VM *pVM, const char *pText)

Evaluates a block of input text in the context of the specified interpreter. Emits any requested output to the interpreter's output function. If the input string is NULL terminated, you can pass -1 as nChars rather than count it. Execution returns when the text block has been executed, or an error occurs. Returns one of the VM_XXXX codes defined in ficl.h: VM_OUTOFTEXT is the normal exit condition. VM_ERREXIT means the interpreter encountered a syntax error and the VM has been reset to recover. VM_USEREXIT means the user executed the "bye" command. VM_ABORT and VM_ABORTQ are generated by 'abort' and 'abort"'. ficlExec calls can be nested, and the function itself is re-entrant, but note that a VM is static, so you have to take reasonable precautions (for example, use one VM per thread in a multithreaded system if you want multiple threads to be able to execute commands). If you call ficlExec() or one of its brothers, you MUST ensure pVM->sourceID was set to a sensible value. ficlExec() explicitly DOES NOT manage SOURCE-ID for you.

int ficlExecC(FICL_VM *pVM, const char *pText, FICL_INT nChars)

Same as ficlExec, but takes a count indicating the length of the supplied string. Setting nChars to -1 is equivalent to ficlExec (expects '\0' termination).

int ficlExecXT(FICL_VM *pVM, FICL_WORD *pWord)

Same as ficlExec, but takes a pointer to a FICL_WORD instead of a string. Executes the word and returns after it has finished. If executing the word results in an exception, this function will re-throw the same code if it is nested under another ficlExec family function, or return the exception code directly if not. This function is useful if you need to execute the same word repeatedly - you save the dictionary search and outer interpreter overhead.

FICL_VM *ficlNewVM(FICL_SYSTEM *pSys)

Create a new VM from the heap, and link it into the system VM list. Initializes the VM and binds default sized stacks to it. Returns the address of the VM, or NULL if an error occurs.

void ficlFreeVM(FICL_VM *pVM)

Force deletion of a VM. You do not need to do this unless you're creating and discarding a lot of VMs. For systems that use a constant pool of VMs for the life of the system, ficlTermSystem takes care of VM cleanup automatically.

void vmInterrupt(FICL_VM *pVM)

Interrupt a running VM from an external context such as a signal handler, hardware interrupt, watchdog timer ISR, or another thread. Causes the VM to longjmp back to the nearest exception recovery point with VM_INTERRUPT, which the caller of ficlExec (or ficlExecXT) will receive as a return code. This is how the console application in testmain.c handles Ctrl-C, for example. This is also the recommended way to break out of infinite loops or long-running computations (assuming they are coded in Ficl) in embedded systems: configure a watchdog orperiodic timer to call vmInterrupt on the active VM, and handle the VM_INTERRUPT return code in your outer loop. Safe to call from POSIX signal handlers on targets that use siglongjmp.

int ficlSetStackSize(int nStackCells)

Set the stack sizes (return and parameter) to be used for all subsequently created VMs. Returns actual stack size to be used.

FICL_WORD *ficlLookup(FICL_SYSTEM *pSys, const char *name)

Returns the address of the most recently defined word in the system dictionary with the given name, or NULL if no match. The address is also known as an XT and can be used in a call to ficlExecXT.

FICL_DICT *ficlGetDict(FICL_SYSTEM *pSys)

Returns a pointer to the main system dictionary, or NULL if the system is uninitialized.

FICL_HASH *ficlGetEnv(FICL_SYSTEM *pSys)

Returns a pointer to the environment wordlist (FICL_HASH). This wordlist is allocated within the main dictionary and stores implementation information as required by the Standard. The returned value is a wordlist identifier (wid) that can be added to the search order, enabling direct execution of environment constants.

void ficlSetEnv(FICL_SYSTEM *pSys, const char *name, FICL_UNS value)

Enters a new constant into the environment wordlist, with the specified name and value. The word is allocated in the main dictionary but inserted into the environment wordlist.

void ficlSetEnvD(FICL_SYSTEM *pSys, const char *name, FICL_UNS hi, FICL_UNS lo)

Enters a new double-cell constant into the environment wordlist with the specified name and value.

void ficlSetEnvF(FICL_SYSTEM *pSys, const char *name, FICL_FLOAT f)

Enters a new floating-point constant into the environment wordlist with the specified name and value. Only available when FICL_WANT_FLOAT is enabled.

FICL_DICT *ficlGetLoc(FICL_SYSTEM *pSys)

Returns a pointer to the locals dictionary. This function is defined only if FICL_WANT_LOCALS is #defined as non-zero (see sysdep.h). The locals dictionary is the symbol table for local variables.

int ficlBuild(FICL_SYSTEM *pSys, const char *name, FICL_CODE code, char flags)

Builds a word into the system default dictionary in a thread-safe way. Parameters: name -- the name of the word to be built; code -- code to execute when the word is invoked (must take a single pointer to a FICL_VM); flags -- 0 or more of FW_IMMEDIATE, FW_COMPILE, use bitwise OR (most words can use FW_DEFAULT).

void ficlCompileCore(FICL_SYSTEM *pSys)

Builds the ANS CORE wordset into the dictionary - called by ficlInitSystem. No need to call it again.

void ficlCompilePrefix(FICL_SYSTEM *pSys)

Builds the prefix parse step words into the dictionary.

void ficlCompileSearch(FICL_SYSTEM *pSys)

Builds the SEARCH-ORDER wordset into the dictionary.

void ficlCompileSoftCore(FICL_SYSTEM *pSys)

Defined in softcore.c, this function builds ANS required words and ficl extras by evaluating a text string. Python3 script softcore.py generates the string in softcore.c from .fr files in ficl/softcore.

void ficlCompileTools(FICL_SYSTEM *pSys)

Builds the TOOLS and TOOLS EXT wordsets into the dictionary.

void ficlCompileFile(FICL_SYSTEM *pSys)

Builds the File-Access wordset into the dictionary.

void ficlCompileFloat(FICL_SYSTEM *pSys)

Builds the optional FLOAT wordset into the dictionary. Only available when FICL_WANT_FLOAT is enabled.

void ficlCompilePlatform(FICL_SYSTEM *pSys)

Builds platform-specific extension words into the dictionary. Only available when FICL_PLATFORM_EXTEND is enabled.

Ficl Internals

Major Data Structures

A running memory image of Ficl consists of one or more FICL_SYSTEMs, each of which owns exactly one dictionary (FICL_DICT), and one or more virtual machines (FICL_VM). Each VM owns two stacks (FICL_STACK) - one for parameters (the parameter stack) and one for return addresses (the return stack). Ficl is a permissive, untyped language by nature, so its fundamental unit of storage is a CELL: a chunk of memory large enough to hold an address or a scalar type.

FICL_SYSTEM

FICL_DICT

FICL_VM

FICL_STACK

Inner Interpreter: Example Execution

This example shows how the inner interpreter executes a simple colon definition with control flow. The word below computes a factorial using a counted loop and a running accumulator.

: fact ( n -- n! )
  1 swap 1+ 1 ?DO
    I *
  LOOP
;

Ficl stores colon definitions as a payload of execution tokens (XTs). In simplified form, the payload for fact looks like this:

XT (literal) 1
XT swap
XT (literal) 1
XT +
XT (?do) [exit address]
XT I
XT *
XT (loop) [branch address]
XT (;)

When C code calls ficlExecXT with the fact word, it sets up a nested exception frame, pushes the exit-inner sentinel on the IP stack, and then enters the inner loop. The inner loop walks the XT list until (;) triggers VM_INNEREXIT.

If an XT in the payload refers to a colon definition, the inner loop executes its code like any other word. The callee pushes the current instruction pointer, switches to its own payload, and returns to the caller when (;) pops the saved IP.

• Setup Save the current pState and runningWord, install a new jmp_buf, and push pExitInner onto the IP stack.
• Execute vmExecute positions the IP at the word's payload, then vmInnerLoop executes XTs in order.
• Looping (?do) initializes the loop parameters, I fetches the loop index, and (loop) advances or exits the loop.
• Exit (;) throws VM_INNEREXIT, the inner loop unwinds, and ficlExecXT restores the previous pState.

Ficl extras

Number syntax

ok> decimal 123 . cr
123
ok> 0x123 . cr
291
ok> 0x123 x. cr
123

The SEARCH wordset and Ficl extensions

Ficl implements many of the search order words in terms of two primitives called >SEARCH and SEARCH>. As their names suggest (assuming you're familiar with Forth), they push and pop the search order stack.

The standard does not appear to specify any conditions under which the search order is reset to a sane state. Ficl resets the search order to its default state whenever ABORT happens. This includes stack underflows and overflows. QUIT does not affect the search order. The minimum search order (set by ONLY) is equivalent to

FORTH-WORDLIST 1 SET-ORDER

There is a default maximum of 16 wordlists in the search order. This can be changed by redefining FICL_DEFAULT_VOCS (declared in sysdep.h).

Note: Ficl resets the search order whenever it does ABORT. If you don't like this behavior, just comment out the dictResetSearchOrder() lines in ficlExec().

>search ( wid -- )

Push wid onto the search order. Many of the other search order words are written in terms of the SEARCH> and >SEARCH primitives. This word can be defined in ANS Forth as follows

: >search   >r get-order 1+ r> swap set-order ;

search>   ( -- wid )

Pop wid off the search order (can be coded in ANS Forth as : search>  get-order nip 1- set-order ; )

ficl-set-current   ( wid -- old-wid )

Set wid as compile wordlist, leaving the previous compile wordlist on the stack

ficl-vocabulary   ( nBins "name" -- )

Creates a ficl-wordlist with the specified number of hash table bins, binds it to the name, and associates the semantics of vocabulary with it (replaces the top wid in the search order list with its own wid when executed)

ficl-wordlist   ( nBins -- wid )

Creates a wordlist with the specified number of hash table bins, and leaves the address of the wordlist on the stack. A ficl-wordlist behaves exactly as a regular wordlist, but it may search faster depending on the number of bins chosen and the number of words it contains at search time. As implemented in ficl, a wordlist is single threaded by default. ficl-named-wordlist takes a name for the wordlist and creates a word that pushes the wid. This is by contrast to VOCABULARY, which also has a name, but replaces the top of the search order with its wid.

forget-wid   ( wid -- )

Iterates through the specified wordlist and unlinks all definitions whose xt addresses are greater than or equal to the value of HERE, the dictionary fill pointer. 

hide   ( -- current-wid-was )

Push the hidden wordlist onto the search order, and set it as the current compile wordlist (unsing ficl-set-current). Leaves the previous compile wordlist ID. I use this word to hide implementation factor words that have low reuse potential so that they don't clutter the default wordlist. To undo the effect of hide, execute  previous set-current

hidden   ( -- wid )

Wordlist for storing implementation factors of ficl provided words. To see what's in there, try:  hide words previous set-current

wid-get-name   ( wid -- c-addr u )

Ficl wordlists have a name property that can be assigned. This is used by ORDER to list the names of wordlists in the search order. 

wid-set-name   ( c-addr wid -- )

Ficl wordlists have a name property that can be assigned. This is used by ORDER to list the names of wordlists in the search order. The name is assumed to be a \0 terminated string (C style), which conveniently is how Ficl stores word names.  See softwords/softcore.fr definition of brand-wordlist 

wid-set-super   ( wid -- )

Ficl wordlists have a parent wordlist pointer that is not specified in standard Forth. Ficl initializes this pointer to NULL whenever it creates a wordlist, so it ordinarily has no effect. This word sets the parent pointer to the wordlist specified on the top of the stack. Ficl's implementation of SEARCH-WORDLIST will chain backward through the parent link of the wordlist when searching. This simplifies Ficl's object model in that the search order does not need to reflect an object's class hierarchy when searching for a method. It is possible to implement Ficl object syntax in strict ANS Forth, but method finders need to manipulate the search order explicitly.

User variables

user   ( -- ) name

Create a user variable with the given name. User variables are virtual machine local. Each VM allocates a fixed amount of storage for them. You can change the maximum number of user variables allowed by defining FICL_USER_CELLS on your compiiler's command line. Default is 16 user cells. User variables behave like VARIABLEs in all other respects (you use @ and ! on them, for example). Example:

user current-class

0 current-class !

Miscellaneous

-roll   ( xu xu-1 ... x0 u -- x0 xu-1 ... x1 ) 

Rotate u+1 items on top of the stack after removing u. Rotation is in the opposite sense to ROLL

-rot   ( a b c -- c a b )

Rotate the top three stack entries, moving the top of stack to third place. I like to think of this as 11/2swap because it's good for tucking a single cell value behind a cell-pair (like an object). 

.dict   ( -- )

Print dictionary usage statistics

.env   ( -- )

List all environment variables provided by the system

environment-wordlist   ( -- wid )

Returns the wordlist identifier for the environment query wordlist. This wordlist can be added to the search order (e.g. environment-wordlist >search) to allow direct execution of environment constants.

.hash   ( -- )

List hash table performance statistics of the wordlist that's first in the search order

.ver   ( -- )

Display ficl version ID

>buf   ( xt c-addr -- )

Redirect VM text output to the null-terminated buffer at c-addr, execute xt to completion, then restore the previous output handler and pExtend pointer. The buffer is cleared to an empty string before xt runs. The caller is responsible for ensuring the buffer is large enough to hold all text produced by xt. Example:

: greet ." Hello, World!" ;
create mybuf 256 allot
' greet mybuf >buf   ( capture greet's output into mybuf )
mybuf type cr         ( prints: Hello, World! )

>name   ( xt -- c-addr u )

Convert a word's execution token into the address and length of its name

body>   ( a-addr -- xt )

Reverses the effect of CORE word >body (converts a parameter field address to an execution token)

compile-only

Mark the most recently defined word as being executable only while in compile state. Many immediate words have this property.

empty   ( -- ) 

Empty the parameter stack

endif

Synonym for THEN

last-word   ( -- xt )

Pushes the xt address of the most recently defined word. This applies to colon definitions, constants, variables, and words that use create. You can print the name of the most recently defined word with 

last-word >name type 

parse-word   ( <spaces>name -- c-addr u )

Skip leading spaces and parse name delimited by a space. c-addr is the address within the input buffer and u is the length of the selected string. If the parse area is empty, the resulting string has a zero length. (From the Standard)

q@   ( addr -- x )

Fetch a 32 bit quantity from the specified address

q!   ( x addr -- )

Store a 32 bit quantity to the specified address 

w@   ( addr -- x )

Fetch a 16 bit quantity from the specified address

w!   ( x addr -- )

Store a 16 bit quantity to the specified address (the low 16 bits of the given value)

x.   ( x -- )

Pop and display the value in hex format, regardless of the current value of BASE

Extra words defined in testmain.c

break   ( -- )

Does nothing - just a handy place to set a debugger breakpoint

cd      ( "directory-name<newline>" -- )

Changes the working directory using the platform chdir() function (POSIX or Win32).

clock   ( -- now )

Returns the number of high-resolution timer ticks elapsed since process start (monotonic on POSIX/macOS, QPC on Windows).

clocks/sec   ( -- clocks_per_sec )

Pushes the number of ticks in a second for clock (1,000,000,000 on POSIX/macOS; QPC frequency on Windows).

load    ( "filename<newline>" -- )

Opens the Forth source file specified and loads it one line at a time, like INCLUDED (FILE)

pwd     ( -- )

Prints the current working directory as set by cd

system  ( "command<newline>" -- )

Issues a command to the platform shell using the standard C system() call.

spewhash   ( "filename<newline>" -- )

Dumps all threads of the current compilation wordlist to the specified text file. This was useful when I thought there might be some point in attempting to optimize the hash function. I no longer harbor those illusions.

test-error   ( -- )

Increments the internal test-failure counter. Called by ficltest.fr whenever a test assertion fails. Not intended for interactive use.

#errors   ( -- n )

Pushes the cumulative test-failure count (as incremented by test-error) onto the stack. Used by the test harness to report the total number of failures after a test run.

Local Variables

Johns-Hopkins local variables

{ <initialized-locals> | <cleared-locals> -- <ignored> }

: local-demo  { a b | c -- }
   ." a = " a . cr
   ." b = " b . cr
   ." c = " c . cr ;
1 2 local-demo  ( you should see 1 2 0 )

: my2dup  { 2:x }  x x ;  ( uses a 2local )
1 2 my2dup .s
.( you should see 1 2 1 2 on the stack ) cr empty

: my2swap  { 2:x 2:y -- y x }  y x ;  ( note use of 2locals )
1 2 3 4 my2swap .s
.( you should see 3 4 1 2 on the stack ) cr empty

: totally-lame-swap  { x y | temp -- y x }
   y to temp
   x to y
   temp to x
   x y ;

\ Using a float local (requires FICL_WANT_FLOAT)
: circle-area  { f:r -- f:area }
  r r f* 3.14159265e f* ;

Standard LOCALS and LOCALS EXT words

Ficl implements both local variable syntaxes suggested in DPANS Appendix A.13. Examples:

\ Using LOCALS| from LOCALS EXT
: -rot  ( a b c -- c a b )
  locals| c b a |
  c a b
;

\ Using LOCAL END-LOCAL
: -rot  ( a b c -- c a b )
   local c
   local b
   local a
   end-locals
   c a b
;

Build Controls

Local variable support is optional because it adds a small amount of overhead to the outer interpreter. You can disable it by setting FICL_WANT_LOCALS to 0 in sysdep.h or in your compiler flags.

Notes:

• Ficl's local variable syntax can make code quite a bit easier to read, so I encourage you to experiment with it.
• Locals or not, keep your words simple. If the stack twiddles are too complex, consider factoring the definition into smaller and simpler words. 
• Ficl's OOP code makes heavy use of local variables, so if you enable FICL_WANT_OOP, locals will also be enabled automatically.
• The default maximum number of local variables per definition is 16. It's controlled by FICL_MAX_LOCALS in sysdep.h. 

Object Oriented Programming

( instance-addr class-addr )

ONLY   ( reset to default search order )
ALSO OOP DEFINITIONS

metaclass --> methods

object --> sub c-led
c-byte obj: .state
: init   { 2:this -- }
    this --> super --> init
    ." initializing an instance of "
    this --> class --> id type cr ;
: on   { led# 2:this -- }
    this --> .state --> get
    1 led# lshift or dup !led
    this --> .state --> set  ;
: off   { led# 2:this -- }
    this --> .state --> get
    1 led# lshift invert and dup !led
    this --> .state --> set  ;
end-class

c-led --> new led

led --> methods
led --> pedigree
1 led --> on
1 led --> off

led  2dup  --> methods  --> pedigree

c-led --> see init

led --> class --> see init

object --> sub c1
: m1   { 2:this -- }  ." c1's m1" cr ;
: m2   { 2:this -- }  ." Running  " this  my=> m1 ; ( early )
: m3   { 2:this -- }  ." Running  " this --> m1     ( late )
end-class
c1     --> sub c2
: m1   { 2:this -- }  ." c2's m1" cr ;
end-class
c2 --> new i2
i2 --> m1   ( runs the m1 defined in c2 )
i2 --> m2   ( is this what you wanted? )
i2 --> m3   { runs the overridden m1)

: new   \ ( class metaclass "name" -- )
    metaclass => instance --> init ;

metaclass --> see new

object subclass c-wordlist
  c-wordlist  ref: .parent
  c-ptr       obj: .name
  c-cell      obj: .size
  c-word      ref: .hash

: ?
  2drop ." ficl wordlist" cr ;
: push  drop >search ;
: pop   2drop previous ;
: set-current drop set-current ;
: words  --> push  words previous ;
end-class

: named-wid   ( "name" -- )
   wordlist  postpone c-wordlist --> ref ;

wordlist  c-wordlist --> ref my-wordlist

my-wordlist --> push
my-wordlist --> set-current
order

c-string --> new homer
s" In this house, " homer --> set
s" we obey the laws of thermodynamics!" homer --> cat
homer --> type

object subclass c-example
  cell:         .cell0
  c-4byte  obj: .nCells
 4 c-4byte array: .quad
  char:         .length
79 chars:       .name
end-class

Parse Steps

Overview

Ficl 2.05 and later includes an extensible parser chain. Ficl feeds every incoming token (chunk of text with no internal whitespace) to each step in the parse chain in turn. The first parse step that successfully matches the token applies semantics to it and returns a TRUE flag, ending the sequence. If all parse steps fire without a match, ficl prints an error message and resets the virtual machine. Parse steps can be written in precompiled code, or in ficl itself, and can be appended to the chain at run-time if you like.

More detail:

• If compiling and local variable support is enabled, attempt to find the token in the local variable dictionary. If found, execute the token's compilation semantics and return
• Attempt to find the token in the system dictionary. If found, execute the token's semantics (may be different when compiling than when interpreting) and return
• If prefix support is enabled (Compile-time constant FICL_WANT_PREFIX in sysdep.h is non-zero), attempt to match the beginning of the token to the list of known prefixes. If there's a match, execute the associated prefix method.
• Attempt to convert the token to a number in the present BASE. If successful, push the value onto the stack if interpreting, compile it if compiling. Return
• All previous parse steps failed to recognize the token. Print " not found" and abort

Adding Parse Steps

my-parse-step   ( c-addr u -- ??? flag )

: ?silly   ( c-addr u -- flag )
   ." Oh no! Not another  " type cr  true ;
' ?silly add-parse-step
parse-order

The other way to add a parse step is by writing it in C, and inserting it into the parse chain with:

void ficlAddPrecompiledParseStep(FICL_SYSTEM *pSys, char *name, FICL_PARSE_STEP pStep);

typedef int (*FICL_PARSE_STEP)(FICL_VM *pVM, STRINGINFO si);

Upon entry to the parse step, si points to the incoming token. The parse step must return FICL_TRUE if it succeeds in matching the token, and FICL_TRUE otherwise. If it succeeds in matching a token, the parse step applies semantics to it before returning. See ficlParseNumber() in words.c for an example.

Adding Prefixes

What's a prefix, anyway? A prefix (contributed by Larry Hastings) is a token that's recognized as the beginning of another token. Its presence modifies the semantics of the rest of the token. An example is 0x, which causes digits following it to be converted to hex regardless of the current value of BASE.

Caveat: Prefixes are matched in sequence, so the more of them there are, the slower the interpreter gets. On the other hand, because the prefix parse step occurs immediately after the dictionary lookup step, if you have a prefix for a particular purpose, using it may save time since it stops the parse process.

Each prefix is a ficl word stored in a special wordlist called <prefixes>. When the prefix parse step (?prefix AKA ficlParsePrefix()) fires, it searches each word in <prefixes> in turn, comparing it with the initial characters of the incoming token. If a prefix matches, the parse step returns the remainder of the token to the input stream and executes the code associated with the prefix. This code can be anything you like, but it would typically do something with the remainder of the token. If the prefix code does not consume the rest of the token, it will go through the parse process again (which may be what you want).

Prefixes are defined in prefix.c and in softwords/prefix.fr. The easiest way to add a new prefix is to insert it into prefix.fr and rebuild the system. You can also add prefixes interactively by bracketing prefix definitions as follows (see prefix,fr):

start-prefixes  ( defined in prefix.fr )
\ make dot-paren a prefix (create an alias for it in the prefixes list)
: .(  .( ;
: 0b  2 __tempbase ; immediate
end-prefixes

The precompiled word __tempbase is a helper for prefixes that specify a temporary value of BASE.

Constant FICL_EXTENDED_PREFIX controls the inclusion of a bunch of additional prefix definitions. This is turned off in the default build since several of these prefixes alter standard behavior, but you might like them.

Notes

Prefixes and parser extensions are non-standard, although with the exception of prefix support, ficl's default parse order follows the standard. Inserting parse steps in some other order will almost certainly break standard behavior.

The number of parse steps that can be added to the system is limited by the value of FICL_MAX_PARSE_STEPS (defined in sysdep.h unless you define it first), which defaults to 8. More parse steps means slower average interpret and compile performance, so be sparing. Same applies to the number of prefixes defined for the system, since each one has to be matched in turn before it can be proven that no prefix matches. On the other hand, if prefixes are defined, use them when possible: since they are matched early in the parse order, a prefix match short circuits the parse process, saving time relative to (for example) using a number builder parse step at the end of the parse chain.

Compile time constant FICL_EXTENDED_PREFIX enables several more prefix definitions in prefix.c and prefix.fr. Please note that this will slow average compile and interpret speed in most cases.

Parser Glossary

parse-order ( -- )

Prints the list of parse steps in the order in which they are evaluated. Each step is the name of a ficl word with the following signature:

parse-step   ( c-addr u -- ??? flag )

A parse step consumes a counted string (the incoming token) from the stack, and exits leaving a flag on top of the stack (it may also leave other parameters as side effects). The flag is true if the parse step succeeded at recognizing the token, false otherwise.

add-parse-step ( xt -- )

Appends a parse step to the parse chain. XT is the adress (execution token) of a ficl word to use as the parse step. The word must have the following signature:

parse-step   ( c-addr u -- ??? flag )

A parse step consumes a counted string (the incoming token) from the stack, and exits leaving a flag on top of the stack (it may also leave other parameters as side effects). The flag is true if the parse step succeeded at recognizing the token, false otherwise.

show-prefixes ( -- )

Defined in softwords/prefix.fr. Prints the list of all prefixes. Each prefix is a ficl word that is executed if its name is found at the beginning of a token. See softwords/prefix.fr and prefix.c for examples.

start-prefixes ( -- )

Defined in softwords/prefix.fr. Declares the beginning of one or more prefix definitions (it just switches the compile wordlist to <prefixes>

end-prefixes ( -- )

Defined in softwords/prefix.fr. Restores the compilation wordlist that was in effect before the last invocation of start-prefixes. Note: the prior wordlist ID is stored in a Ficl variable, so attempts to nest start-prefixes end-prefixes blocks wil result in mildly silly side effects.

Debugger

Ficl includes a simple step debugger for colon definitions and does> words. If you use it and can suggest improvements (or better yet if you write some), please let me know.

Using the debugger

debug <your word here>

If the word is unnamed, or all you have is an xt, you can instead use:

The debugger invokes see on the word, printing a crude source listing, then stops at the first instruction of the definition. There are four (case insensitive) commands you can use from here onwards:

I (step in)

If the next instruction is a colon defintion or does> word, steps into that word's code. If the word is a primitive, simply executes the word.

O (step over)

Executes the next instruction in its entirety

G (go)

Run the word to completion and exit the debugger

L (list)

Lists the source code of the word presently being stepped

Q (quit)

Abort the word and exit the debugger, clearing the stack

X (eXecute)

Interpret the remainder of the line as ficl words for their side effects. Any errors will abort the debug session and reset the VM. Usage example:

x drop 3 \ fix argument on stack

Anything else

Prints a list of available debugger commands

The on-step event

If there is a defined word named on-step when the debugger starts, that word will be executed before every step. As a guideline, this word should have no side effects. Its intended use is to display the stacks and any other VM state you're interested in, but you may have some better ideas. If so, please let me know. The default on-step is:

: on-step ." S: " .s cr ;

Other useful words for debugging and on-step

r.s ( -- )

Prints a represention of the state of the return stack non-destructively. You have to have a good understanding of the return stack side-effects of control words to make sense of it, but it does give an accurate representation of what's there. Example: DO .. LOOPs stack three parameters on the return stack: the loop count and limit, and the LEAVE target address.

.s ( -- )

Prints the parameter stack non-destructively

f.s ( -- )

Prints the float stack non-destructively (only available if FICL_WANT_FLOAT is enabled)

Debugger internals

The debugger words are mostly located in source file tools.c. There are supporting words (debug and on-step) in softcore.fr as well. There are two main words that make the debugger go: debug-xt and step-break. Debug-xt takes the xt of a word to debug (as returned by ', for example) checks to see if it is debuggable (not a primitive), sets a breakpoint at its first instruction, and runs see on it. To set a breakpoint, debug-xt replaces the instruction at the breakpoint with the xt of step-break, and stores the original instruction and its address in a static breakpoint record. To clear the breakpoint, step-break simply replaces the original instruction and adjusts the target virtual machine's instruction pointer to run it.

Step-break is responsible for processing debugger commands and setting breakpoints at subsequent instructions.

To Do

• The debugger needs to exit automatically when it encounters the end of the word it was asked to debug. Perhaps this could be a special kind of breakpoint?
• Add user-set breakpoints
• Add "step out" command

ANS Required Information

• ANS Forth System
• Providing names from the Core Extensions word set 
• Providing the Exception word set
• Providing names from the Exception Extensions word set
• Providing the Locals word set 
• Providing the Locals Extensions word set 
• Providing the Memory Allocation word set
• Providing the Programming-Tools word set
• Providing names from the Programming-Tools Extensions word set
• Providing the Search-Order word set
• Providing the Search-Order Extensions word set

Implementation-defined Options

• aligned address requirements (3.1.3.3 Addresses); 
  System dependent. You can change the default address alignment by defining FICL_ALIGN on your compiler's command line. The default value is set to 2 in sysdep.h. This causes dictionary entries and ALIGN and ALIGNED to align on 4 byte boundaries. To align on 2ⁿ byte boundaries, set FICL_ALIGN to n. 
• behavior of 6.1.1320 EMIT for non-graphic characters; 
  Depends on target system, C runtime library, and your implementation of ficlTextOut(). 
• character editing of 6.1.0695 ACCEPT and 6.2.1390 EXPECT; 
  None implemented in the versions supplied in words.c. Because ficlExec() is supplied a text buffer externally, it's up to your system to define how that buffer will be obtained.
• character set (3.1.2 Character types, 6.1.1320 EMIT, 6.1.1750 KEY); 
  Depends on target system and implementation of ficlTextOut().
• character-aligned address requirements (3.1.3.3 Addresses); 
  Ficl characters are one byte each. There are no alignment requirements.
• character-set-extensions matching characteristics (3.4.2 Finding definition names); 
  No special processing is performed on characters beyond case-folding. Therefore, extended characters will not match their unaccented counterparts.
• conditions under which control characters match a space delimiter (3.4.1.1 Delimiters); 
  Ficl uses the Standard C function isspace() to distinguish space characters. The rest is up to your library vendor. 
• format of the control-flow stack (3.2.3.2 Control-flow stack); 
  Uses the data stack
• conversion of digits larger than thirty-five (3.2.1.2 Digit conversion); 
  The maximum supported value of BASE is 36. Ficl will assertion fail in function ltoa of vm.c if the base is found to be larger than 36 or smaller than 2. There will be no effect if NDEBUG is defined, however, other than possibly unexpected behavior. 
• display after input terminates in 6.1.0695 ACCEPT and 6.2.1390 EXPECT; 
  Target system dependent
• exception abort sequence (as in 6.1.0680 ABORT"); 
  Does ABORT 
• input line terminator (3.2.4.1 User input device); 
  Target system dependent (implementation of outer loop that calls ficlExec) 
• maximum size of a counted string, in characters (3.1.3.4 Counted strings, 6.1.2450 WORD); 
  255 
• maximum size of a parsed string (3.4.1 Parsing); 
  Limited by available memory and the maximum unsigned value that can fit in a CELL (2³²-1). 
• maximum size of a definition name, in characters (3.3.1.2 Definition names); 
  Ficl stores the first 31 characters of a definition name.
• maximum string length for 6.1.1345 ENVIRONMENT?, in characters; 
  Same as maximum definition name length 
• method of selecting 3.2.4.1 User input device; 
  None supported. This is up to the target system 
• method of selecting 3.2.4.2 User output device; 
  None supported. This is up to the target system 
• methods of dictionary compilation (3.3 The Forth dictionary); 
• number of bits in one address unit (3.1.3.3 Addresses); 
  Target system dependent. Ficl usually supports processors that can address 8 bit quantities, but there is no dependency that I'm aware of. 
• number representation and arithmetic (3.2.1.1 Internal number representation); 
  System dependent. Ficl represents a CELL internally as a union that can hold INT32 (a signed 32 bit scalar value), UNS32 (32 bits unsigned), and an untyped pointer. No specific byte ordering is assumed. 
• ranges for n, +n, u, d, +d, and ud (3.1.3 Single-cell types, 3.1.4 Cell-pair types); 
  Assuming a 32 bit implementation, range for signed single-cell values is -2³¹..2³¹-1. Range for unsigned single cell values is 0..2³²-1. Range for signed double-cell values is -2⁶³..2⁶³-1. Range for unsigned single cell values is 0..2⁶⁴-1. 
• read-only data-space regions (3.3.3 Data space);
  None 
• size of buffer at 6.1.2450 WORD (3.3.3.6 Other transient regions); 
  Default is 255. Depends on the setting of nPAD in ficl.h. 
• size of one cell in address units (3.1.3 Single-cell types); 
  System dependent, typically four for 32 bit targets, 8 for 64 bit targets.
• size of one character in address units (3.1.2 Character types); 
  System dependent, usually one.
• size of the keyboard terminal input buffer (3.3.3.5 Input buffers); 
  This buffer is supplied by the host program. Ficl imposes no practical limit. 
• size of the pictured numeric output string buffer (3.3.3.6 Other transient regions); 
  Default is 255 characters. Depends on the setting of nPAD in ficl.h. 
• size of the scratch area whose address is returned by 6.2.2000 PAD (3.3.3.6 Other transient regions); 
  Not presently supported 
• system case-sensitivity characteristics (3.4.2 Finding definition names); 
  Ficl is not case sensitive 
• system prompt (3.4 The Forth text interpreter, 6.1.2050 QUIT); 
  "ok>" 
• type of division rounding (3.2.2.1 Integer division, 6.1.0100 */, 6.1.0110 */MOD, 6.1.0230 /, 6.1.0240 /MOD, 6.1.1890 MOD); 
  Symmetric rounding (toward zero) 
• values of 6.1.2250 STATE when true; 
  One (no others) 
• values returned after arithmetic overflow (3.2.2.2 Other integer operations); 
  System dependent. Ficl makes no special checks for overflow. 
• whether the current definition can be found after 6.1.1250 DOES> (6.1.0450 :). 
  No. Definitions are unsmudged after ; only, and only then if no control structure matching problems have been detected. 

Ambiguous Conditions

The following general ambiguous conditions could occur because of a combination of factors: 

• a name is neither a valid definition name nor a valid number during text interpretation (3.4 The Forth text interpreter); 
  Ficl does ABORT and prints the name followed by " not found". 
• a definition name exceeded the maximum length allowed (3.3.1.2 Definition names); 
  Ficl stores the first 31 characters of the definition name, and uses all characters of the name in computing its hash code. The actual length of the name, up to 255 characters, is stored in the definition's length field. 
• addressing a region not listed in 3.3.3 Data Space; 
  No problem: all addresses in ficl are absolute. You can reach any address in Ficl's address space.
• argument type incompatible with specified input parameter, e.g., passing a flag to a word expecting an n (3.1 Data types); 
  Ficl makes no check for argument type compatibility. Effects of a mismatch vary widely depending on the specific problem and operands. 
• attempting to obtain the execution token, (e.g., with 6.1.0070 ', 6.1.1550 FIND, etc.) of a definition with undefined interpretation semantics; 
  Ficl returns a valid token, but the result of executing that token while interpreting may be undesirable.
• dividing by zero (6.1.0100 */, 6.1.0110 */MOD, 6.1.0230 /, 6.1.0240 /MOD, 6.1.1561 FM/MOD, 6.1.1890 MOD, 6.1.2214 SM/REM, 6.1.2370 UM/MOD, 8.6.1.1820 M*/);
  Results are target procesor dependent. Usually, Ficl makes no check for divide-by-zero. The target processor will throw an exception. 
• insufficient data-stack space or return-stack space (stack overflow); 
  With FICL_ROBUST (sysdep.h) set >= 2, most parameter stack operations are checked for underflow and overflow. Ficl does not check the return stack. 
• insufficient space for loop-control parameters; 
  No check - Evil results. 
• insufficient space in the dictionary; 
  Ficl generates an error message if the dictionary is too full to create a definition header. It checks ALLOT as well, but it is possible to make an unchecked allocation request that overflows the dictionary. 
• interpreting a word with undefined interpretation semantics; 
  Ficl protects all ANS Forth words with undefined interpretation semantics from being executed while in interpret state. It is possible to defeat this protection using ' (tick) and EXECUTE, though. 
• modifying the contents of the input buffer or a string literal (3.3.3.4 Text-literal regions, 3.3.3.5 Input buffers); 
  Varies depending on the nature of the buffer. The input buffer is supplied by ficl's host function, and may reside in read-only memory. If so, writing the input buffer can ganerate an exception. String literals are stored in the dictionary, and are writable. 
• overflow of a pictured numeric output string;
  In the unlikely event you are able to construct a pictured numeric string of more than 255 characters, the system will be corrupted unpredictably. The buffer area that holds pictured numeric output is at the end of the virtual machine. Whatever is mapped after the offending VM in memory will be trashed, along with the heap structures that contain it. 
• parsed string overflow;
  Ficl does not copy parsed strings unless asked to. Ordinarily, a string parsed from the input buffer during normal interpretation is left in-place, so there is no possibility of overflow. If you ask to parse a string into the dictionary, as in SLITERAL, you need to have enough room for the string, otherwise bad things may happen. This is not usually a problem. 
• producing a result out of range, e.g., multiplication (using *) results in a value too big to be represented by a single-cell integer (6.1.0090 *, 6.1.0100 */, 6.1.0110 */MOD, 6.1.0570 >NUMBER, 6.1.1561 FM/MOD, 6.1.2214 SM/REM, 6.1.2370 UM/MOD, 6.2.0970 CONVERT, 8.6.1.1820 M*/); 
  Value will be truncated to fit in a single cell. 
• reading from an empty data stack or return stack (stack underflow); 
  Most stack underflows are detected and prevented if FICL_ROBUST (sysdep.h) is set to 2 or greater. Otherwise, the stack pointer and size are likely to be trashed. 
• unexpected end of input buffer, resulting in an attempt to use a zero-length string as a name; 
  Ficl returns for a new input buffer until a non-empty one is supplied. 

• >IN greater than size of input buffer (3.4.1 Parsing)
  Bad Things occur - unpredictable bacause the input buffer is supplied by the host program's outer loop. 
• 6.1.2120 RECURSE appears after 6.1.1250 DOES>
  It finds the address of the definition before DOES>
• argument input source different than current input source for 6.2.2148 RESTORE-INPUT
  Not implemented 
• data space containing definitions is de-allocated (3.3.3.2 Contiguous regions)
  This is OK until the cells are overwritten with something else. The dictionary maintains a hash table, and the table must be updated in order to de-allocate words without corruption. 
• data space read/write with incorrect alignment (3.3.3.1 Address alignment)
  Target processor dependent. Consequences include: none (Intel), address error exception (68K). 
• data-space pointer not properly aligned (6.1.0150 ,, 6.1.0860 C,)
  See above on data space read/write alignment 
• less than u+2 stack items (6.2.2030 PICK, 6.2.2150 ROLL)
  Ficl detects a stack underflow and reports it, executing ABORT, as long as FICL_ROBUST is two or larger. 
• loop-control parameters not available ( 6.1.0140 +LOOP, 6.1.1680 I, 6.1.1730 J, 6.1.1760 LEAVE, 6.1.1800 LOOP, 6.1.2380 UNLOOP)
  Loop initiation words are responsible for checking the stack and guaranteeing that the control parameters are pushed. Any underflows will be detected early if FICL_ROBUST is set to two or greater. Note however that Ficl only checks for return stack underflows at the end of each line of text. 
• most recent definition does not have a name (6.1.1710 IMMEDIATE)
  No problem. 
• name not defined by 6.2.2405 VALUE used by 6.2.2295 TO
  Ficl's version of TO works correctly with VALUEs, CONSTANTs and VARIABLEs. 
• name not found (6.1.0070 ', 6.1.2033 POSTPONE, 6.1.2510 ['], 6.2.2530 [COMPILE])
  Ficl prints an error message and does ABORT
• parameters are not of the same type (6.1.1240 DO, 6.2.0620 ?DO, 6.2.2440 WITHIN)
  No check. Results vary depending on the specific problem. 
• 6.1.2033 POSTPONE or 6.2.2530 [COMPILE] applied to 6.2.2295 TO
  The word is postponed correctly. 
• string longer than a counted string returned by 6.1.2450 WORD
  Ficl stores the first FICL_STRING_MAX-1 chars in the destination buffer. (The extra character is the trailing space required by the standard. Yuck.) 
• u greater than or equal to the number of bits in a cell (6.1.1805 LSHIFT, 6.1.2162 RSHIFT)
  Depends on target process or and C runtime library implementations of the << and >> operators on unsigned values. For I386, the processor appears to shift modulo the number of bits in a cell. 
• word not defined via 6.1.1000 CREATE (6.1.0550 >BODY, 6.1.1250 DOES>)
• words improperly used outside 6.1.0490 <# and 6.1.0040 #> (6.1.0030 #, 6.1.0050 #S, 6.1.1670 HOLD, 6.1.2210 SIGN)
  Don't. CREATE reserves a field in words it builds for DOES>to fill in. If you use DOES> on a word not made by CREATE, it will overwrite the first cell of its parameter area. That's probably not what you want. Likewise, pictured numeric words assume that there is a string under construction in the VM's scratch buffer. If that's not the case, results may be unpleasant.

Locals Implementation-defined options

• maximum number of locals in a definition (13.3.3 Processing locals, 13.6.2.1795 LOCALS|)
  Default is 16. Change by redefining FICL_MAX_LOCALS, defined in sysdep.h

Locals Ambiguous conditions

• executing a named local while in interpretation state (13.6.1.0086 (LOCAL))
  Locals can be found in interpretation state while in the context of a definition under construction. Under these circumstances, locals behave correctly. Locals are not visible at all outside the scope of a definition. 
• name not defined by VALUE or LOCAL (13.6.1.2295 TO)
  See the CORE ambiguous conditions, above (no change)

Programming Tools Implementation-defined options

• source and format of display by 15.6.1.2194 SEE
  SEE de-compiles definitions from the dictionary. Because Ficl words are threaded by their header addresses, it is very straightforward to print the name and other characteristics of words in a definition. Primitives are so noted. Colon definitions are decompiled, but branch target labels are not reconstructed. Literals and string literals are so noted, and their contents displayed.

Search Order Implementation-defined options

• maximum number of word lists in the search order (16.3.3 Finding definition names, 16.6.1.2197 SET-ORDER) 
  Defaults to 16. Can be changed by redefining FICL_DEFAULT_VOCS, declared in sysdep.h 
• minimum search order (16.6.1.2197 SET-ORDER, 16.6.2.1965 ONLY) 
  Equivalent to FORTH-WORDLIST 1 SET-ORDER

Search Order Ambiguous conditions

• changing the compilation word list (16.3.3 Finding definition names)
  Ficl stores a link to the current definition independently of the compile wordlist while it is being defined, and links it into the compile wordlist only after the definition completes successfully. Changing the compile wordlist mid-definition will cause the definition to link into the new compile wordlist. 
• search order empty (16.6.2.2037 PREVIOUS)
  Ficl prints an error message if the search order underflows, and resets the order to its default state. 
• too many word lists in search order (16.6.2.0715 ALSO)
  Ficl prints an error message if the search order overflows, and resets the order to its default state.