]> VCONIC

ghostofbasho@gmail.com

Applications and Real-Time vCon Internet-Draft This document defines the World Transcription Format (WTF) extension for Virtualized Conversations (vCon). The WTF extension provides a standardized analysis framework for representing speech-to-text transcription data from multiple providers within vCon containers. This extension defines a comprehensive analysis format that enables consistent transcription processing, quality assessment, and interoperability across different transcription services while preserving provider-specific features through extensible fields. The WTF extension is designed as a Compatible Extension that introduces a new analysis type for transcription data without altering existing vCon semantics, ensuring backward compatibility with existing vCon implementations. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the vCon Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 RFC2119 when, and only when, they appear in all capitals, as shown here.

Introduction Virtualized Conversations (vCon) provide a standardized container format for conversation data, enabling interoperability across different communication platforms and modalities. An overview of the vCon ecosystem and its use cases is provided in . As speech-to-text technology becomes increasingly important for conversation analysis, compliance, and accessibility, there is a growing need for standardized transcription analysis representation within vCon containers. Current transcription services each use proprietary formats, making it difficult to:

• Switch between transcription providers without data conversion
• Perform comparative analysis across different providers
• Maintain consistent data processing pipelines
• Preserve transcription metadata during system migrations
• Ensure long-term accessibility and archival compliance

This document defines the World Transcription Format (WTF) extension for vCon, which addresses these challenges by providing a comprehensive analysis framework that includes:

• A unified analysis schema for transcription data from any speech-to-text provider
• Hierarchical organization from words to segments to complete transcripts
• Extensible fields for provider-specific analytical features
• Built-in quality metrics and confidence scoring for analysis assessment
• Support for real-time and batch transcription analysis workflows
• Consistent export capabilities to standard subtitle and caption formats

The WTF extension defines a new category of vCon analysis specifically for speech-to-text transcription. This analysis type enables organizations to standardize their transcription analysis workflows while maintaining the flexibility to use multiple providers and preserve provider-specific analytical enhancements.

Conventions and Definitions

Core Terms World Transcription Format (WTF): A standardized analysis framework and JSON schema for representing speech-to-text transcription data from any provider in a consistent, interoperable format. WTF defines a comprehensive analysis type for vCon that includes transcription content, quality metrics, confidence scoring, and provider-specific analytical features. Transcription Analysis: The structured representation of speech-to-text conversion results, including the transcribed text, timing information, confidence metrics, speaker identification, and quality assessments. Transcription Provider: A service or system that converts audio or video content to text, such as Whisper(TM), Deepgram(TM), AssemblyAI(TM), Google Cloud Speech-to-Text(TM), Amazon Transcribe(TM), or Azure Speech Services(TM). Segment: A logical chunk of transcribed content, typically representing sentence or phrase boundaries with associated timing information and analytical metadata. Speaker Diarization: The process of partitioning an audio stream into homogeneous segments according to the speaker identity, enabling "who spoke when" analysis. Compatible Extension: A vCon extension that introduces additional data without altering the meaning or structure of existing elements, as defined in .

World Transcription Format Overview The World Transcription Format defines a comprehensive analysis framework using a hierarchical JSON structure designed to capture transcription analysis results at multiple levels of granularity while maintaining consistency across different providers.

Design Principles Completeness: WTF captures all essential transcription analysis data including text, timing, confidence scores, speaker information, and quality assessments. Consistency: Provides uniform analysis structure regardless of the underlying transcription provider. Extensibility: Supports provider-specific analytical features through structured extension fields. Validation: Includes built-in data integrity checking and quality metrics for analysis assessment. Hierarchical Organization: Natural progression from words to segments to complete transcripts enables multi-level analysis.

Core Structure WTF uses a hierarchical JSON structure with three required sections and multiple optional enrichment layers:

vCon WTF Extension Definition

Extension Classification The WTF extension is a Compatible Extension as defined in Section 2.5 of . This extension:

• Introduces a new analysis type for transcription data without altering existing vCon semantics
• Defines a comprehensive analysis framework for speech-to-text results
• Can be safely ignored by implementations that don't support transcription analysis
• Does not require listing in the critical parameter
• Maintains backward compatibility with existing vCon implementations

Extension Registration This document defines the "wtf_transcription" extension token for registration in the vCon Extensions Names Registry:

• Extension Name: wtf_transcription
• Extension Description: World Transcription Format - A standardized analysis framework for speech-to-text transcription with multi-provider support, quality metrics, and confidence scoring
• Change Controller: IESG
• Specification Document: This document

Extension Usage vCon instances that include WTF transcription analysis SHOULD include "wtf_transcription" in the extensions array to indicate support for the WTF analysis framework:

WTF Analysis Structure

Analysis Storage Transcription analysis results MUST be stored as vCon analysis using the standard analysis object structure defined in Section 4.5 of . The analysis mechanism provides the association between the analysis results and the corresponding dialog elements. The WTF transcription analysis object MUST include:

• type: MUST be set to "wtf_transcription" to identify this as a WTF analysis
• encoding: MUST be set to "json" for structured analysis data
• body: MUST contain the WTF transcription analysis data structure as defined below

The WTF transcription analysis SHOULD include:

• start: ISO 8601 timestamp when the transcription analysis was performed
• party: Index of the party in the vCon parties array (for single-speaker transcription analysis)
• dialog: Index of the associated dialog in the vCon dialog array

WTF Analysis Schema The body field of the WTF transcription analysis object MUST contain a JSON object conforming to the WTF analysis schema with the following structure:

Required Fields

Transcript Object The transcript object provides the high-level summary of the entire transcription:

• text: The complete, concatenated transcription text
• language: MUST use BCP-47 format (examples: "en-US", "es-MX", "fr-CA")
• duration: Floating-point seconds, MUST be >= 0
• confidence: Normalized to [0, 1] range regardless of provider scale

Segments Array Segments represent logical chunks of transcribed content, typically sentence or phrase boundaries:

• id: MUST be unique within the document, typically sequential
• start/end: Floating-point seconds, where end > start
• text: SHOULD be trimmed of leading/trailing whitespace
• speaker: Can be integer (0, 1, 2) or string ("Speaker A")
• words: References indices in the words array

Metadata Object The metadata object captures processing and source information:

• created_at/processed_at: MUST use ISO 8601 format
• provider: Lowercase identifier for supported providers
• model: Provider's model identifier (e.g., "whisper-large-v3", "nova-2")
• options: Preserves provider-specific configuration

Optional Fields

Words Array

Speakers Object

Quality Object

Extensions Object

Alternatives Array The alternatives array captures multiple transcription hypotheses when a provider returns ranked results.

Enrichments Object The enrichments object contains NLP analysis results layered on top of the base transcription.

Streaming Object The streaming object captures metadata specific to real-time transcription sessions.

Provider Integration Guidelines

Supported Providers The WTF extension supports integration with major transcription providers:

• Whisper(TM): OpenAI's open-source speech recognition system
• Deepgram(TM): Real-time speech-to-text API
• AssemblyAI(TM): AI-powered transcription and audio intelligence
• Google Cloud Speech-to-Text(TM): Google's speech recognition service
• Amazon Transcribe(TM): AWS speech-to-text service
• Azure Speech Services(TM): Microsoft's speech recognition platform
• Rev.ai(TM): Automated and human transcription services
• Speechmatics(TM): Real-time and batch speech recognition
• Wav2Vec2(TM): Facebook's self-supervised speech recognition model
• Parakeet(TM): NVIDIA's speech recognition toolkit

Conversion Requirements When converting from provider-specific formats to WTF:

1. Normalize confidence scores to [0.0, 1.0] range
2. Convert timestamps to floating-point seconds
3. Standardize language codes to BCP-47 format
4. Preserve provider-specific features in extensions field
5. Validate output against WTF schema requirements

Provider-Specific Mappings

Whisper Integration

Deepgram Integration

Quality and Confidence Metrics

Confidence Score Normalization All confidence scores MUST be normalized to the [0.0, 1.0] range:

• 1.0: Highest confidence (perfect recognition)
• 0.9-1.0: High confidence
• 0.7-0.9: Medium confidence
• 0.5-0.7: Low confidence
• 0.0-0.5: Very low confidence

Provider-specific scales are converted during import:

• Percentage (0-100) -> divide by 100
• Log probability -> exponential transformation
• Custom scales -> linear normalization

Quality Metrics The quality object provides assessment metrics:

• audio_quality: Categorical assessment (high/medium/low)
• background_noise: Noise level [0.0-1.0]
• multiple_speakers: Boolean indicator of multi-speaker content
• overlapping_speech: Boolean indicator of speaker overlap
• silence_ratio: Percentage of audio that is silence
• average_confidence: Mean confidence across all words/segments
• low_confidence_words: Count of words below 0.5 confidence
• processing_warnings: Array of processing issues or notices

Security Considerations

Data Privacy Transcription data often contains sensitive personal information. Developers implementing WTF SHOULD consult for comprehensive guidance on privacy considerations when handling vCon data. Implementations SHOULD:

• Apply appropriate access controls to WTF analysis objects
• Consider encryption requirements for transcription data at rest and in transit
• Implement data retention policies consistent with privacy regulations
• Provide mechanisms for transcription data redaction or anonymization

Provider-Specific Security When integrating with external transcription providers:

• Validate provider credentials and API security
• Implement secure communication channels (TLS 1.2 or higher)
• Consider data residency requirements for audio processing
• Audit provider data handling practices and compliance certifications

Integrity Protection WTF transcription analysis objects SHOULD be integrity protected using vCon signing mechanisms as defined in to prevent unauthorized modification of transcription data.

Temporal Validation Implementations SHOULD validate that transcription timestamps are consistent with the associated dialog timing information to detect potential tampering or synchronization issues.

WTF as an Analysis Framework The World Transcription Format is fundamentally an analysis framework that defines how speech-to-text transcription results should be structured, validated, and stored within vCon containers. This section clarifies the distinction between WTF as an analysis type and its storage mechanism.

Analysis vs. Storage WTF defines an analysis type: The core contribution of this specification is the definition of a standardized analysis framework for transcription data. This includes:

• Structured representation of transcription content (words, segments, complete transcripts)
• Quality assessment metrics (confidence scores, audio quality indicators)
• Speaker diarization analysis
• Provider-specific analytical features
• Processing metadata and provenance information

Analysis provides storage: The vCon analysis mechanism is used as the storage container for WTF analysis results. The analysis type "wtf_transcription" identifies the stored data as conforming to the WTF analysis schema.

Analysis Processing Workflow The typical workflow for WTF transcription analysis is:

1. Audio Processing: Dialog audio is processed by a transcription provider
2. Analysis Generation: Provider results are converted to WTF analysis schema
3. Quality Assessment: Analysis includes confidence scores and quality metrics
4. Storage: WTF analysis is stored as a vCon analysis object with type "wtf_transcription"
5. Retrieval: Consumers read the analysis object to access transcription results
6. Utilization: Analysis data is used for search, compliance, accessibility, or further processing

IANA Considerations

vCon Extensions Names Registry This document requests IANA to register the following extension in the vCon Extensions Names Registry established by :

• Extension Name: wtf_transcription
• Extension Description: World Transcription Format - A standardized analysis framework for speech-to-text transcription with multi-provider support, quality metrics, and confidence scoring
• Change Controller: IESG
• Specification Document: This document

WTF Analysis Type Values Registry This document requests IANA to establish a new registry for WTF analysis type values with the following initial registration:

• Type Value: wtf_transcription
• Description: Structured speech-to-text transcription analysis using the World Transcription Format framework, including transcription content, quality metrics, confidence scoring, and provider-specific analytical features
• Change Controller: IESG
• Specification Document: This document

Registration Template

• Type Value: The string value used as the analysis type identifier in vCon analysis objects
• Description: Brief description of the analysis type and its analytical capabilities
• Change Controller: For Standards Track RFCs, list "IESG". For others, give the name of the responsible party.
• Specification Document(s): Reference to defining documents with URIs where available

WTF Provider Registry This document requests IANA to establish a new registry for WTF transcription analysis providers with initial registrations for supported providers:

• Provider Name: whisper
• Description: OpenAI Whisper(TM) speech recognition system
• Change Controller: IESG
• Specification Document: This document (Additional provider registrations would be added for each supported provider)

Examples

Basic Two-Party Call Transcription Analysis

Multi-Provider Transcription Analysis Comparison

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. SIPez LLC Informative References VCONIC Marashlian & Donahue, PLLC VCONIC

Acknowledgements

• Appreciation to the transcription provider community for their input on standardization requirements and analysis framework design.
• Thanks to the vCon working group for their feedback and guidance on extension design patterns and the distinction between analysis types and attachment mechanisms.

Trademark Notice All trademarks mentioned in this document are the property of their respective owners. The use of these trademarks does not imply endorsement by the IETF or the authors of this document. The following trademarks are referenced:

• Whisper(TM) is a trademark of OpenAI, Inc.
• Deepgram(TM) is a trademark of Deepgram, Inc.
• AssemblyAI(TM) is a trademark of AssemblyAI, Inc.
• Google Cloud Speech-to-Text(TM) is a trademark of Google LLC.
• Amazon Transcribe(TM) is a trademark of Amazon.com, Inc.
• Azure Speech Services(TM) is a trademark of Microsoft Corporation.
• Rev.ai(TM) is a trademark of Rev.com, Inc.
• Speechmatics(TM) is a trademark of Speechmatics Limited.
• Wav2Vec2(TM) is a trademark of Meta Platforms, Inc.
• Parakeet(TM) is a trademark of NVIDIA Corporation.