cloud34221.autos

Getting Started with eXist-db: A Beginner’s Guide

Written by

admin

in

Building a RESTful API with eXist-db and XQuery RESTXQThis article walks through building a production-ready RESTful API using eXist-db (an XML-native database) and XQuery RESTXQ. It covers architecture, project setup, API design, request handling with RESTXQ annotations, data modeling, authentication basics, pagination and filtering, error handling, testing, deployment, and performance considerations. Code samples use XQuery 3.1 and assume eXist-db 5.x or later.

Why eXist-db + RESTXQ?

  • eXist-db is an XML-native database with built-in web application support, full-text search, XQuery processing, and REST interfaces.
  • RESTXQ is a lightweight annotation-based mechanism to expose XQuery functions as HTTP endpoints — no separate application server required.
  • This combo is ideal when your data model is XML or you need tight integration between storage and query logic.

Architecture overview

A typical architecture for an eXist-db + RESTXQ API:

  • Client (browser, mobile, or other services)
  • eXist-db instance hosting:
    • XML collections as persistent storage
    • XQuery modules exposing REST endpoints via RESTXQ
    • Optional app code for authentication, caching, and utilities
  • Reverse proxy (NGINX) for TLS, routing, rate-limiting
  • Optional caching (Varnish, Redis) and load balancing for scaling

Project structure

A recommended collection layout inside eXist-db (stored under /db/apps/myapi):

  • /db/apps/myapi/config/ — config files, e.g., app.xconf
  • /db/apps/myapi/modules/ — XQuery modules
    • api.xqm — main RESTXQ module
    • model.xqm — data access helpers
    • auth.xqm — authentication helpers
    • util.xqm — common utilities
  • /db/apps/myapi/data/ — sample XML documents or initial data
  • /db/apps/myapi/static/ — static assets for documentation or UI

Store this structure in the filesystem during development and deploy using eXide, the dashboard, or eXist’s package manager.

Data modeling

Design your XML representation to fit the domain and queries. Example: a simple “article” resource.

Example article document (article-0001.xml):

<article id="0001" xmlns="http://example.org/ns/article">   <title>Introduction to eXist-db</title>   <author>     <name>Jane Doe</name>     <email>[email protected]</email>   </author>   <published>2024-05-01</published>   <tags>     <tag>xml</tag>     <tag>database</tag>   </tags>   <content>     <p>eXist-db is an open-source native XML database...</p>   </content> </article>

Store articles inside a collection, e.g., /db/apps/myapi/data/articles/.

Design considerations:

  • Use stable, human-readable IDs when possible.
  • Keep namespaces consistent.
  • Normalize repeated data (authors as separate collection) only when it simplifies updates.

REST API design

Define resources and endpoints. Example routes:

  • GET /api/articles — list articles (with pagination & filtering)
  • POST /api/articles — create new article
  • GET /api/articles/{id} — retrieve article
  • PUT /api/articles/{id} — update article
  • DELETE /api/articles/{id} — delete article
  • GET /api/articles/search?q=… — full-text search

Use standard HTTP status codes and content negotiation (XML and JSON responses).

Using RESTXQ: basics

RESTXQ exposes XQuery functions as HTTP endpoints using annotations. Save modules under modules/ and declare the RESTXQ namespace.

Basic RESTXQ module skeleton (api.xqm):

xquery version "3.1"; module namespace api="http://example.org/api"; import module namespace rest="http://expath.org/ns/restxq"; declare   %rest:path("/api/articles")   %rest:GET function api:get-articles() {   (: implementation :) };

RESTXQ annotations you’ll use most:

  • %rest:path — path template (supports {var} placeholders)
  • %rest:GET, %rest:POST, %rest:PUT, %rest:DELETE — HTTP methods
  • %rest:produces, %rest:consumes — content types
  • %rest:param — query/path parameters
  • %rest:status — default status code

Implementing CRUD operations

Below are concise, working examples for each operation. These assume helper functions from model.xqm to read/write documents.

model.xqm (helpers):

xquery version "3.1"; module namespace model="http://example.org/model"; declare option exist:serialize "method=xml"; (: Get all article nodes :) declare function model:get-all-articles() {   collection('/db/apps/myapi/data/articles')/article }; (: Get single article by @id :) declare function model:get-article($id as xs:string) {   collection('/db/apps/myapi/data/articles')/article[@id=$id][1] }; (: Save or replace an article document :) declare function model:save-article($doc as node()) {   let $path := concat('/db/apps/myapi/data/articles/article-', $doc/@id, '.xml')   return xmldb:store('/db/apps/myapi', 'data/articles', fn:concat('article-', $doc/@id, '.xml'), $doc) }; (: Delete article :) declare function model:delete-article($id as xs:string) {   let $res := xmldb:remove('/db/apps/myapi', 'data/articles', fn:concat('article-', $id, '.xml'))   return $res };

API module (api.xqm):

xquery version "3.1"; module namespace api="http://example.org/api"; import module namespace rest="http://expath.org/ns/restxq"; import module namespace model="http://example.org/model"; declare   %rest:path("/api/articles")   %rest:GET   %rest:produces("application/xml") function api:list-articles() {   <articles>{     for $a in model:get-all-articles()     return $a   }</articles> }; declare   %rest:path("/api/articles/{id}")   %rest:GET   %rest:produces("application/xml")   %rest:param("id", "{id}") function api:get-article($id as xs:string) {   let $art := model:get-article($id)   return     if ($art) then $art     else (       rest:set-response-status(404),       <error>Article not found</error>     ) }; declare   %rest:path("/api/articles")   %rest:POST   %rest:consumes("application/xml")   %rest:produces("application/xml") function api:create-article($body as element()) {   let $id := string($body/@id)   return (     model:save-article($body),     rest:set-response-status(201),     $body   ) }; declare   %rest:path("/api/articles/{id}")   %rest:PUT   %rest:consumes("application/xml")   %rest:produces("application/xml")   %rest:param("id","{id}") function api:update-article($id as xs:string, $body as element()) {   return (     model:save-article($body),     rest:set-response-status(200),     $body   ) }; declare   %rest:path("/api/articles/{id}")   %rest:DELETE   %rest:param("id","{id}") function api:delete-article($id as xs:string) {   let $res := model:delete-article($id)   return     if ($res) then rest:set-response-status(204) else rest:set-response-status(404) };

Notes:

  • xmldb:store and xmldb:remove are eXist-specific XQuery functions for document management.
  • For JSON support, you can produce application/json by serializing XML to JSON using e.g., json:serialize-from-xml or manual conversion.

Content negotiation (XML + JSON)

To support both XML and JSON, inspect the Accept header and return accordingly. Simplest approach: provide separate endpoints or use %rest:produces with multiple types and a negotiation helper.

Example accept helper:

declare function util:accepts-json() as xs:boolean {   contains(lower-case(request:get-accept-header()), 'application/json') };

Then convert XML to JSON when needed:

let $xml := model:get-article($id) return if (util:accepts-json()) then json:serialize-from-xml($xml) else $xml

Pagination and filtering

Implement query params: page, pageSize, q (search), tag filters.

Example:

declare   %rest:path("/api/articles")   %rest:GET   %rest:param("page")   %rest:param("pageSize") function api:list-articles($page as xs:integer? := 1, $pageSize as xs:integer? := 10) {   let $start := (($page - 1) * $pageSize) + 1   let $items := model:get-all-articles()   let $paged := subsequence($items, $start, $pageSize)   return <articles page="{ $page }" pageSize="{ $pageSize }" total="{ count($items) }">{ $paged }</articles> };

For full-text, use eXist’s ft: or contains-text functions:

  • ft:query
  • xmldb:fulltext or use XQuery Full Text features.

Authentication & authorization (basic)

eXist supports user accounts, role checks, and you can implement token-based auth in XQuery.

Simple API-key example (not production-secure):

  • Store API keys in /db/apps/myapi/config/apikeys.xml
  • Write auth.xqm to verify X-API-Key header against stored keys
  • Annotate protected endpoints to call auth:require-api-key() at the top

auth.xqm:

module namespace auth="http://example.org/auth"; declare function auth:require-api-key() {   let $key := request:get-header("X-API-Key")   let $valid := doc('/db/apps/myapi/config/apikeys.xml')//key[text() = $key]   return if ($valid) then true() else (rest:set-response-status(401), error()) };

Place auth:require-api-key() inside functions before sensitive operations.

For production use: use OAuth2/OpenID Connect via a reverse proxy or implement JWT verification in XQuery.

Error handling and responses

  • Use HTTP status codes: 200, 201, 204, 400, 401, 403, 404, 500.
  • Return machine-readable error payloads: 
    
<error> <code>400</code> <message>Invalid payload: missing title</message> </error>
  • Use try/catch in XQuery for predictable error mapping: 
    
try { (: risky :) } catch * { rest:set-response-status(500), <error>{ string(.) }</error> }

Testing

  • Unit test XQuery modules using eXist’s unit testing framework or third-party tools.
  • End-to-end: use curl, httpie, Postman, or automated tests (pytest + requests).
  • Example curl create: curl -X POST -H “Content-Type: application/xml” –data-binary @article.xml “https://api.example.com/api/articles”

Logging and monitoring

  • Use eXist’s logging (log4j) configuration to write application logs.
  • Emit structured logs for requests and errors.
  • Monitor with Prometheus (exporter) and use alerting for error rates and latency.

Caching & performance

  • Cache expensive queries in eXist with in-memory collections or use external caches (Redis, Varnish).
  • Use indexes: eXist supports range, full-text, and token indexes. Define indexes in collection configuration to speed queries.
  • Avoid large document write contention; shard collections or partition by time/ID when scaling writes.

Deployment

  • Bundle as an eXist app (EAR-like package) or use the package manager.
  • Front eXist with NGINX for TLS and rate-limiting.
  • For high availability, run multiple eXist nodes behind a load balancer and use shared storage or replication.

Example: Adding JSON input support for create

To accept JSON input for creating an article, parse it and convert to XML:

declare   %rest:path("/api/articles")   %rest:POST   %rest:consumes("application/json") function api:create-article-json($body as document-node()) {   let $json := json:parse(request:get-body())   let $xml :=     <article id="{ $json?id }" xmlns="http://example.org/ns/article">       <title>{ $json?title }</title>       <author>         <name>{ $json?author?name }</name>         <email>{ $json?author?email }</email>       </author>       <published>{ $json?published }</published>       <content>{ $json?content }</content>     </article>   return (model:save-article($xml), rest:set-response-status(201), $xml) };

Security considerations

  • Validate and sanitize inputs to prevent XML injection and XQuery injection.
  • Use least privilege for eXist users and collections.
  • Protect admin endpoints and config files; store secrets outside world-readable collections.
  • Prefer JWT/OAuth behind a secure reverse proxy for production auth.

Summary

Building a RESTful API with eXist-db and RESTXQ gives you a compact, integrated platform when your domain uses XML. RESTXQ keeps routing and handlers in XQuery, minimizing context switching. Key steps: design XML schema, create model helpers, expose endpoints with annotations, handle content negotiation, add auth and error handling, and tune with indexes and caching.

For hands-on: set up a small eXist app using the examples above, iterate on schema/indexes, and add tests and monitoring before production rollout.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

More posts