]>

rohan.mahy@gmail.com

Applications and Real-Time More Instant Messaging Interoperability mimi content image metadata audio metadata video metadata The More Instant Messaging Interoperability (MIMI) content format is a container for rich content, which can reference image, video, and audio files. This document describes metadata for these files to allow for more pleasant rendering. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the More Instant Messaging Interoperability Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction The MIMI content format can convey a variety of media types, as either inline or referenced external content. In messaging applications it is common to display audio, video, and static image content, collectively audio/video (AV). The layout for messaging applications often reserves a placeholder for the AV content. While it is common for static images to be immediately displayed, audio and video content is often not immediately downloaded and rendered. Even if image data is downloaded immediately, if there is a network or server delay there can be time when the aspect ratio or dimensions of the image are not yet know. It is therefore useful to have some rendering hints about the media for more pleasant rendering. This document defines extensions to the MIMI content format to provide these hints.

Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses a variety of terms from the MIMI content format definition, especially NestedPart, SinglePart, ExternalPart, and MultiPart.

AV Metadata Extensions The AV Metadata MIMI content extension is an array of AV metadata entries. Each AV metadata entry is a CBOR map of AV metadata properties, all of which are optional except the part_index and type. The semantics of the individual property fields is as follows:

• part_index: refers to the order of MIMI parts from the relevant part inside the NestedPart structure in a MIMI content message. It can refer to a SinglePart or ExternalPart.
• type: an integer enumeration representing the media type (not-including the subtype). audio is 1, image is 2, and video is 3. An extension socket is defined, although its need is not anticipated.
• width: the width of the image or video in pixels
• height: the height of the image or video in pixels
• duration: the duration of the audio or video in milliseconds, expressed as an unsigned integer
• preview_index: for a video part, the partIndex of another related part that represents its image preview. It can refer to a SinglePart or ExternalPart, or a MultiPart with chooseOne partSemantics which contains only SinglePart or ExternalPart types, all of which must be an image with a disposition value of preview.
• accessibility_text: this text could be rendered instead of the audio, image, or video when various accessiblity settings are enabled, or during no or slow network access when a cached or preview image is not available.
• rotation: one of four values: 0, 90, 180, or 270. This integer refers to the number of degrees of clockwise rotation (in 90 degree increments) needed to correctly view the image.

• Note that an orientation field is not necessary. Any image and video with a width field which is larger than its height is assumed to have a landscape mode orientation, while one with a height larger than its width is assumed to have a portrait mode orientation.

The following snippet of Concise Data Definition Language (CDDL) is used to formally define the structure of the extension.

Example Below is an example of a video of puppies, a preview image, and an audio clip.

Security Considerations TODO Security

IANA Considerations IANA is requested to add the following entries to the MIMI Content Extension Keys registry.

av-metadata MIMI Content Extension Key The following completed registration template is provided:

• Key: TBD1 (suggested value 6)
• Name: av-metadata
• Type: array
• Recommended: Y
• Reference: of this document

Normative References Rohan Mahy Consulting Services This document describes content semantics common in Instant Messaging (IM) systems and describes a profile suitable for instant messaging interoperability of messages end-to-end encrypted inside the MLS (Message Layer Security) Protocol. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document specifies a new opcode in the Secure Shell terminal modes encoding. The new opcode describes the widely used IUTF8 terminal mode bit, which indicates that terminal I/O uses UTF-8 character encoding.

Acknowledgments TODO acknowledge.