Sitemap, robots and indexing
Indexing controls and crawler-facing files.
Sitemap, robots and indexing
VELORYZ Studio · SEO, analytics and site insights Audience: Marketing collaborators, content leads and developers Guide type: Practical Studio operations guide
Purpose
Sitemap, robots and indexing should be treated as an end-to-end responsibility, not an isolated button. Start by defining what will change, who can approve it and how success will be checked. Indexing controls and crawler-facing files.
The site serves dynamic robots.txt and sitemap.xml. Private/admin routes must remain excluded; all public documentation belongs in the sitemap.
Availability and responsibility
The source generates route metadata, robots.txt and XML sitemap. Any analytics collection, conversion data or external dashboard must be configured and verified before those metrics are reported.
Before committing time or budget, confirm who operates this workflow: the visitor, the client portal user, a Studio owner, or the delivery team. A feature described in design documentation is not automatically a public account entitlement.
Before you begin
- A page-specific search intent and approved title/description copy
- An inventory of canonical public routes and indexing rules
- Consent requirements for analytics and campaign tracking
- A short list of measurable success criteria specifically for sitemap, robots and indexing.
- A recovery plan if the proposed change creates a regression.
Recommended procedure
- Locate the relevant area or public route. For new projects, establish the brief at /create/; for owner-only content changes, use authenticated Studio Control at /admin. Do not assume a missing feature exists.
- Define the specific work item for Sitemap, robots and indexing: capture the initial state, user intent, relevant data, permissions, expected output and what must not change.
- Follow the design and operating decisions listed below. Work in a draft, staging or reviewed project handoff where available; avoid editing live customer data experimentally.
- Review the effect with real content and realistic viewport sizes. Record any exception and obtain the appropriate approval before a public change.
- After the change, verify the public outcome rather than relying on a success message alone; preserve a reference to the reviewed version.
Topic-specific operating procedure
These steps apply to Sitemap, robots and indexing. The implementation must be verified against the current Studio permissions, deployed provider connections and approved project scope.
- Generate sitemap.xml from public canonical routes and hide drafts, admin pages and documentation hidden by CMS.
- Provide a human-readable /sitemap/ for visitors, with all Help Centre and Studio references.
- Serve robots.txt with the absolute XML sitemap address and accurate private-path restrictions.
- Check XML validity, escaping, trailing slash conventions and 200/404 route behavior.
Evidence to retain
- The original input or situation, and a clear before/after comparison.
- The reviewed version, responsible owner and acceptance record.
- Any limitation, follow-up decision, or safely reversible workaround.
Configuration and decision points
- Assign one descriptive title, primary heading and page summary to each public route.
- Keep internal links consistent and generate sitemap entries only for publishable, indexable URLs.
- Test metadata, canonical URL, social sharing and structured data against the real host.
- Configure and verify event analytics only when a provider and consent flow actually exist.
When more than one implementation is possible, prefer the simplest approach that remains accessible, maintainable and compatible with the current source. Note any cost, licensing, privacy, ownership and performance implications before approval.
Verification checklist
- Pages return 200 for published content and 404 for missing paths.
- Canonical paths and XML sitemap agree; private admin and API routes are excluded.
- Search does not return unpublished or hidden documentation entries.
- No performance or conversion numbers are invented from missing tracking data.
Also confirm that the result addresses the original objective: Indexing controls and crawler-facing files.. Review the customer-visible language, error states, navigation paths and the relevant device widths. A green build or a successful save does not by itself establish a correct production outcome.
Troubleshooting and recovery
- If a page is not indexed, verify robots, canonical, HTTP status and content usefulness.
- If social previews are wrong, inspect page metadata and image availability.
- If events are missing, confirm consent, provider configuration and network requests.
When asking for assistance, include the exact public URL or project reference, a concise reproduction path, the expected behaviour, actual behaviour, relevant timestamps and a redacted screenshot if useful. Never include passwords, access tokens or private customer records.
Related reading
- [SEO overview](/docs/studio/seo/overview/)
- [Page titles, descriptions and social previews](/docs/studio/seo/metadata/)
- [Analytics and events](/docs/studio/analytics/overview/)
- [All VELORYZ documentation](/docs/)
- [Studio documentation](/docs/studio/)
- [Help Centre](/help/studio/)
- [/create/](/create/)
- [/docs/overview/](/docs/overview/)
Operational note
This article describes the Studio workflow and its acceptance criteria. Where a self-service control is not available, the action is coordinated with the Studio team rather than silently represented as automated.
