Full Text Search Query Syntax

[resplin]

Obsolete Pages{{Obsolete}}

The official documentation is at: http://docs.alfresco.com

3.3
--briana 10:10, 25 May 2010 (BST)CMIS Query Language
Full Text Search
Search

Please see the the official documentation for the Alfresco Full Text Search syntax for the authoritative source of this information.

The Alfresco Full Text Search (FTS) query text can be used standalone or embedded in CMIS-SQL using the contains() predicate function.
The CMIS specification supports a subset of Alfresco FTS. The full power of Alfresco FTS can not be used and maintain portablity between CMIS repositories.

Alfresco Full Text Search is exposed directly by Share which adds its own template also used as its default field.
The examples provided on the Search page apply to Alfresco FTS. For exmaple, PATH and TYPE queries are the same.
The main difference is the required escaping and that the leading '@' for properties is optional. '@' and ':' do not need to be escaped; the FTS parser can work it out.

The default template (see Full_Text_Search_Query_Syntax#Query_Templates) for Share V3.4 is:

%(cm:name cm:title cm:description ia:whatEvent ia:descriptionEvent lnk:title lnk:description TEXT)

Embedded in CMIS-SQL, only CMIS-SQL style property identifiers (cmis:name) and aliases, CMIS-SQL column aliases and the special fields listed can be used to identify fields. The SQL query defines tables and table aliases after 'from' and 'join' clauses.  If the SQL query references more than one table, the contains() function must specify a single table to use by its alias. All properties in the embedded FTS query are to this table and all column aliases used in the FTS query must refer to the same table. For a single table, the table alias is not required as part of the contains() function.

Used standalone, fields can also be identified using prefix:local-name and {uri}local-name styles.

Table of Contents

• 1 Search for a single term


• 2 Search for a phrase


• 3 Search for an exact term


• 4 Term expansion


• 5 Conjunctions


• 6 Disjunctions


• 7 Negation


• 8 Specifically indicate optional, mandatory, and excluded elements of a query


• 9 Fields


• 10 Wildcards


• 11 Ranges


• 12 Fuzzy matching


• 13 Proximity


• 14 Boosts


• 15 Grouping


• 16 Spans and positions


• 17 Escaping


• 18 Ordering and Control


• 19 Mixed FTS ID behavior


• 20 Operator precedence

Search for a single term

Single terms are tokenized before the search according to the appropriate data dictionary definition(s).
If no field is specified, the default TEXT field is used. This is a shortcut for searching all properties of type content.
If the appropriate data dictionary definition(s) for the field support both FTS and untokenized search, then the FTS search will be used.
FTS will include synonyms if the analyzer generates them.
Terms can not contain whitespace.

banana

TEXT:banana

Both these queries will find any nodes with the word 'banana' in any property of type d:content.

Search for a phrase

Phrases are enclosed in double quotes. Any embedded quotes may be escaped using \'.
If no field is specified, then the default TEXT field will be used, as with searches for a single term.
The whole phrase will be tokenized before the search according to the appropriate data dictionary definition(s).

'big yellow banana'

Search for an exact term

To search for an exact term, prefix the term with '='. This ensures the term will not be tokenized.
If both FTS and ID base search are supported for a specified or implied property, then exact matching will be used where possible.

=running

will match 'running' but will not be tokenized. If you are using stemming, it may not match anything.
For an untokenized property, all will be well.

Term expansion

To force tokenization and term expansion, prefix the term with '~'. For a property with both ID and FTS indexes where the ID index is the default, force the use of the FTS index.

~running

Conjunctions

Single terms, phrases, etc can be combined using 'AND' in upper, lower, or mixed case.

big AND yellow AND banana
TEXT:big TEXT:yellow TEXT:banana
TEXT:big and TEXT:yellow and TEXT:banana

All search for nodes that contain the terms 'big', 'yellow', and 'banana' in any content.

Disjunctions

Single terms, phrases, etc can be combined using 'OR' in upper, lower, or mixed case. If not otherwise specified, the search fragments will be ORed together by default.

big yellow banana
big OR yellow OR banana
TEXT:big OR TEXT:yellow OR TEXT:banana

Negation

Single terms, phrases, etc can be combined using 'NOT' in upper, lower, or mixed case or prefixed with '!' or '-'

yellow NOT banana
yellow !banana
yellow -banana

NOT yellow banana
-yellow banana
!yellow banana

Specifically indicate optional, mandatory, and excluded elements of a query

Sometimes AND and OR are not enough. If you want to find documents that must contain the term 'car', score those with the term 'red' higher, but not match those just containing 'red', you are stuck...

'|'	The field, phrase, group, etc, is optional; a match increases the score.
'+'	The field, phrase, group, etc, is mandatory (this differs from Google - see '=')
'-', '!'	The field, phrase, group, etc, must not match

Returning to our example:

+car |red

finds documents that contain the term 'car', score those with the term 'red' higher, but does not match those just containing 'red'.

At least one element of a query must match (or not match) for there to be any results.

All AND and OR constructs can be expressed with these operators.

Fields

Search specific fields rather than the default. Terms, phrases, etc can all be preceded by a field. If not, the default field TEXT is used.

field:term
field:'phrase'
=field:exact
~field:expand

Fields fall into three types:

• Property fields evaluate the search term against a particular property
  
• Data type fields evaluate the search term against all properties of the given type
  
• Special fields are described as follows:
  

Description	Type	Example	FTS	CMIS contains()
Fully qualified property	Property	{http://www.alfresco.org/model/content/1.0}name:apple	Yes	No
Fully qualified property	Property	@{http://www.alfresco.org/model/content/1.0}name:apple	Yes	No
CMIS style property	Property	cmis:name:apple	Yes (case insensitive)	Yes (case sensitive)
Prefix style property	Property	cm:name:apple	Yes	Yes
Prefix style property	Property	@cm:name:apple	Yes	No
TEXT	Special	TEXT:apple	Yes	3.4.3 onwards
ID	Special	ID:'NodeRef'	Yes	3.4.3 onwards
ISROOT	Special	ISROOT:T	Yes	3.4.3 onwards
TX	Special	TX:'TX'	Yes	3.4.3 onwards
PARENT	Special	PARENT:'NodeRef'	Yes	3.4.3 onwards
PRIMARYPARENT	Special	PRIMARYPARENT:'NodeRef'	Yes	3.4.3 onwards
QNAME	Special	QNAME:'app:company_home'	Yes	3.4.3 onwards
CLASS	Special	CLASS:'qname'	Yes	3.4.3 onwards
EXACTCLASS	Special	EXACTCLASS:'qname'	Yes	3.4.3 onwards
TYPE	Special	TYPE:'qname'	Yes	3.4.3 onwards
EXACTTYPE	Special	EXACTTYPE:'qname'	Yes	3.4.3 onwards
ASPECT	Special	ASPECT:'qname'	Yes	3.4.3 onwards
EXACTASPECT	Special	EXACTASPECT:'qname'	Yes	3.4.3 onwards
ALL	Special	ALL:'text'	Yes	3.4.3 onwards
ISUNSET	Special	ISUNSET:'property-qname'	Yes	3.4.3 onwards
ISNULL	Special	ISNULL:'property-qname'	Yes	3.4.3 onwards
ISNOTNULL	Special	ISNOTNULL:'property-qname'	Yes	3.4.3 onwards
TAG	Special	TAG:'SomeTag'	3.4.3 onwards	3.4.3 onwards
Fully qualified data type	Data Type	{http://www.alfresco.org/model/dictionary/1.0}content:apple	Yes	No
prefixed data type	Data Type	d:content:apple	Yes	No

Note: Most of the special fields not available in CMIS can be expressed in other ways. Type constraints by from, aspects by join, is not null, ID = ..., IN_FOLDER(), ...

Wildcards

Wild cards are supported in both terms, phrases, and exact phrases using '*' to match zero, one or more characters, and '?' to match a single character.
'*' may appear on its own and implies Google style 'anywhere after'
Wildcard pattern can be combined with the '=' prefix for identifier based pattern matching.

TEXT:app?e
TEXT:app*
TEXT:*pple
appl?
*ple
=*ple
'ap*le'
'***le'
'?????'

All the previous examples find the term apple.

Ranges

Inclusive ranges can specified in Google style. There is an extended syntax for more complex ranges.
Unbounded ranges can be defined using MIN and MAX for numeric and date types and '\u0000' and '\FFFF' for text (anything that is invalid will do ....)

Range queries are not supported for all properties of type d:content or d:mltext.
Properties of type d:text are supported. However, if the d:text property is tokenised then you may get unexpected results (unless the property is both tokenised and untokenised in the index).

TEXT by default does not support range queries as it expands to all properties of type d:content.

Lucene	Google	Description	Example
[#1 TO #2]	#1..#2	The range #1 to #2 inclusive #1 0..5 [0 TO 5]
<#1 TO #2]		The range #1 to #2 including #2 but not #1 #1
[#1 TO #2>		The range #1 to #2 including #1 but not #2 #1 [0 TO 5>
<#1 TO #2>		The range #1 to #2 exclusive #1 <0 TO 5>

my:text:apple..banana
my:int:[0 TO 10]
my:float:2.5..3.5
my:float:0..MAX
mt:text:[l TO '\uFFFF']

Fuzzy matching

Fuzzy matching is not currently implemented.
The default lucene implementation is Levenshtein Distance which is expensive to evaluate.

Postfix terms with '~float'

apple~0.8

Proximity

Google style proximity is supported. To specify proximity for fields, you must use grouping.

big * apple
TEXT:(big * apple)
big *(3) apple
TEXT:(big *(3) apple)

Currently unimplemented:

• 'phrase'~3
  
  • phrase slop
    

Boosts

Query time boosts allow matches on certain parts of the query to influence the score more than others. All query elements can be boosted: terms, phrases, exact terms, expanded terms, proximity (only in filed groups), ranges, and groups.

term^2.4
'phrase'^3
term~0.8^4
=term^3
~term^4
cm:name:(big * yellow)^4
1..2^2
[1 TO 2]^2
yellow AND (car OR bus)^3

Grouping

Groupings of terms are made using '(' and ')'. Groupings of all query elements are supported in general.
Groupings are also supported after a field - field group. The query elements in field groups all apply to the same field and can not include a field.

(big OR large) AND banana  
title:((big OR large) AND banana)

Spans and positions

Spans and positions are not currently implemented.
Positions will depend on tokenization. Anything more detailed than
one *(2) two seems arbitrarily dependent on the tokenization.
An identifier and pattern matching, or dual FTS and ID tokenization, may well be the answer in these cases.

Initial ideas:

term[^] - start

term[$] - end

term[position]

These are of possible future use, but excluded for now.

Lucene surround extensions:

and(terms etc)

99w(terms etc )

97n(terms etc)

Escaping

• Any character may be escaped using '\' in TERMS, IDS (field identifiers) and PHRASES
  
• Java unicode escape sequences are supported
  
• Whitespace can be escaped in terms and IDs; the following are supported:
  
  • cm:my\ content:my\ name
    

Ordering and Control

• Ordering and paging
  
  • This is not part of the query language but is available in the API
    

• constraints for field expansion and available fields
  

• context - type/aspect, name space(s)
  

• Set default conjunction or disjunction
  

Mixed FTS ID behavior

priority as defined on properties in the data dictionary

explicit priority by prefixing with '=' for id pattern matches

'~' can be used to force tokenization

Operator precedence

Operator precedence is SQL-like (not Java-like). When there is more than one logical operator in a statement, and they are not explicitly grouped using parentheses, NOT is evaluated first, then AND, and finally OR.

Operator precedence from highest to lowest:

'

[, ],