Skip to main content

Interface: Frame

A single frame, as seen by the camera. This is backed by a C++ HostObject wrapping the native GPU buffer. At a 4k resolution, a Frame can be 1.5MB in size.

Example

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log(`Frame: ${frame.width}x${frame.height} (${frame.pixelFormat})`)
}, [])

Properties​

bytesPerRow​

• Readonly bytesPerRow: number

Returns the amount of bytes per row.

Defined in​

Frame.ts:33

height​

• Readonly height: number

Returns the height of the frame, in pixels.

Defined in​

Frame.ts:29

isMirrored​

• Readonly isMirrored: boolean

Returns whether the Frame is mirrored (selfie camera) or not.

Defined in​

Frame.ts:41

isValid​

• Readonly isValid: boolean

Whether the underlying buffer is still valid or not. A Frame is valid as long as your Frame Processor (or a runAsync(..) operation) is still running

Defined in​

Frame.ts:21

orientation​

• Readonly orientation: Orientation

Represents the orientation of the Frame.

Some ML Models are trained for specific orientations, so they need to be taken into consideration when running a frame processor. See also: isMirrored

Defined in​

Frame.ts:52

pixelFormat​

• Readonly pixelFormat: PixelFormat

Represents the pixel-format of the Frame.

Defined in​

Frame.ts:56

planesCount​

• Readonly planesCount: number

Returns the number of planes this frame contains.

Defined in​

Frame.ts:37

timestamp​

• Readonly timestamp: number

Returns the timestamp of the Frame relative to the host sytem's clock.

Defined in​

Frame.ts:45

width​

• Readonly width: number

Returns the width of the frame, in pixels.

Defined in​

Frame.ts:25

Methods​

toArrayBuffer​

â–¸ toArrayBuffer(): Uint8Array

Get the underlying data of the Frame as a uint8 array buffer. The format of the buffer depends on the Frame's pixelFormat.

Note that Frames are allocated on the GPU, so calling toArrayBuffer() will copy from the GPU to the CPU.

Returns​

Uint8Array

Example

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'

  if (frame.pixelFormat === 'rgb') {
    const data = frame.toArrayBuffer()
    console.log(`Pixel at 0,0: RGB(${data[0]}, ${data[1]}, ${data[2]})`)
  }
}, [])

Defined in​

Frame.ts:76

toString​

â–¸ toString(): string

Returns a string representation of the frame.

Returns​

string

Example

console.log(frame.toString()) // -> "3840 x 2160 Frame"

Defined in​

Frame.ts:84