URI and Query Parameters Encoding

When working with HCL Commerce Search APIs, it is important to understand how URI paths and query parameters are encoded. Proper encoding ensures that requests are interpreted correctly by the server and that special characters are handled consistently across systems. Incorrect and inconsistent encoding can lead to unexpected search results, misinterpreted query parameters, or request errors, especially when working with accented characters.

Understanding URI and Query parameters

A URI (Uniform Resource Identifier) is the part of a URL that identifies the resource being requested. For Example: http://localhost:3737/search/resources/store/1/productview/coffee

Query parameters are additional values included in the URL to provide specific instructions to the server. For Example: ?manufacturer=Le Café

In HCL Commerce Search:

The Search server defaults to decoding URIs using UTF-8
Query parameters may default to ISO-8859-1

To ensure consistent encoding and decoding, always specify: Content-Type: application/json; charset=UTF-8

Modern browsers use UTF-8 for encoding both URIs and query parameters, so explicitly specifying UTF-8 ensures compatibility.

Need for encoding

Proper encoding ensures that characters are transmitted and interpreted correctly across systems.

Accented Characters

Non-English terms such "Café" or "Déco" contain accented characters. If not properly encoded, these characters may not be interpreted correctly. Encoding ensures they are transmitted accurately.

Special Characters

Characters such as spaces, ampersands, and slashes have special meanings in URLs. For example:

A space is encoded as %20

If these characters are not encoded, it may result in incorrect request handling.

Consistency

Encoding provides a standard way to represent characters so that data is interpreted consistently by different clients and servers.

Potential issues without proper encoding

If characters are not properly encoded:

Misterpretation: Special characters may be incorrectly processed by the server, leading to errors or unexpected results.
Data Integrity Issues: Improper encoding may cause the server to receive altered or incorrectly decoded data.

To prevent this issue, always specify:

Content-Type: application/json;
                    charset=UTF-8

Encoding behavior in API requests

UTF-8 Charset Encoding

When encoding URIs and query parameters with UTF-8, the request header must specify the correct charset to avoid decoding issues. For example: http://localhost:3737/search/resources/store/1/productview/bySearchTerm/Caf%C3%A9?manufacturer=Le%20Caf%C3%A9

In this example:

"Café" in the URI path is encoded as Caf%C3%A9
"manufacturer" in the query parameter is encoded as Le%20Caf%C3%A9
The character "é" is represented in UTF-8 as %C3%A9

To ensure both the URI path and query parameter are decoded correctly, specify:

Content-Type: application/json;
                                charset=UTF-8

No Explicit Encoding

Modern clients, such as browsers and tools like Postman, automatically encode URLs and query parameters with UTF-8 by default. For example: http://localhost:3737/search/resources/store/1/productview/bySearchTerm/Café?manufacturer=Le%20Café

In this example:

"Café" is included directly in the URI path.
"manufacturer" is provided as "Le Café" in the query parameter without explicit encoding.
The clients encode these characters in UTF-8, where "é" is represented in UTF-8 as %C3%A9

However, if the server decodes query parameters using ISO-8859-1 and no charset is given, this will result in decoding issues. To ensure both the URI path and query parameter are decoded correctly, specify:

Content-Type: application/json;
                        charset=UTF-8

cURL Example:


curl --location
'http://localhost:3737/search/resources/store/1/productview/bySearchTerm/*?manufacturer=Le%20Caf%C3%A9' \
--header 'Content-Type: application/json; charset=UTF-8'