Unstructured and site content
Unstructured site content includes documents that do not adhere to a specific data model, such as product attachments contained in various formats. For example, content such as user manuals and warranty information are considered unstructured content. Its elements, construction, and organization are typically unknown and can vary depending on its file type.
Although the HCL Commerce database might not store the unstructured content, unstructured content can still be indexed and retrieved. For example, when a search for laptop is submitted, the search result can find the unstructured content such as attachments in .pdf or .doc format, which contain the laptop keyword.
Site content
When you work with search index types, site content is categorized under the catalog entry search index.
Site content includes HTML and other site files from HCL Commerce starter stores. It is fetched and crawled by the site content crawler.
HCL Commerce provides sample static HTML files by default, that the site content crawler fetches and crawls to help populate the site content search index. You can configure the site content crawler to fetch extra content from HCL Commerce starter stores.
For more information, see Indexing site content with HCL Commerce Search.
Supported file types
HCL Commerce Search uses parser libraries to detect and extract metadata and structured text content from documents.
- Microsoft Office
- Excel 97-2003 (.xls).
- JAVA
- Classes (.class).
- Documents and text
- OpenDocument (.odt, odp, .ods).
- Tika 1.7
Unstructured content schema
HCL Commerce Search can directly extract metadata and content from the unstructured data source. Differing unstructured data formats might contain varying metadata information. For example, Microsoft Word files contain metadata such as creator, company, and created date, whereas JPEG image files contain metadata such as width and height.
Solr Cell provides a mechanism to add a prefix to the generated metadata field. This behavior
requests that the typical schema design of unstructured content must contain at least one dynamic
field, such as tika_*
, to store all metadata information. The main difference
between structured and unstructured content is that the name and total number of fields for one
unstructured document might vary from another unstructured document.
HCL Commerce Search manages unstructured content by requesting Tika to parse the documents before they are processed. Then, they are sent to the HCL Commerce Search server for eventual indexing.
Schema changes for related structured and unstructured content
When structured content contains a relationship with unstructured content, it must contain a new field in the structured schema.xml file to represent the unstructured information. This new field can query the structured objects by their unstructured content.
<field name="unstructure" type="wc_text" indexed="true" stored="false" multiValued="true" />
Where
the stored="false"
snippet enables unstructured content to not be retrieved by
queries.