Skip to main content

React Virtual Backgrounds in Standard, Hybrid, and Headless UIs

In ReactJS, virtual backgrounds change the video track that MediaSFU publishes. Standard UI owns the complete workflow. Hybrid UI reuses the SDK modal against the same room engine. Fully custom UI owns the controls while calling the exported processing helpers.

Choose the guide for your SDK​

SDKSupplied background surfaceFully custom boundary
ReactJS 4.4.5ModernBackgroundModal and BackgroundModalapplyVirtualBackground, applyBackgroundBlur, clearVirtualBackground, getLocalVideoStream
Angular 2.5.2BackgroundModal and LaunchBackgroundUse Angular's room/background services; no React helper API
Vue 1.3.2BackgroundModal and useBackgroundModalUse Vue's room state; no React helper API
React Native 2.5.2 and Expo 2.6.2Native background modal displays an unsupported-state messageDo not advertise image or blur processing on native mobile from these packages
Flutter 2.4.4Background controls in the supplied media settingsFlutter background state and platform processor; test each target platform
Android/Kotlin 1.0.9Compose/platform virtual-background processingVirtualBackgroundProcessor and current room state; test on target devices
Swift 0.1.7On-device backgrounds (blur, image, color) in the hosted room's media settingsUse the hosted room; there is no separate Swift background API
Unity preview.2No complete standalone UI workflowUse the package's supported room/media path instead
Shared core 1.3.3No UI; your framework renders the controlsapplyVirtualBackground, applyBackgroundBlur, clearVirtualBackground, getLocalVideoStream with the complete room parameters

The code below is for ReactJS only. For another package, start with its device and background section and SDK guide; do not apply React component props to another UI framework.

Standard: let the room own the feature​

Mount the current complete room and open its media/background settings. The room owns camera acquisition, preview, save, apply, publish, restore, removal, and cleanup. A saved selection can be retained while the camera is off and applied when the room publishes the camera later.

import {ModernMediasfuGeneric} from 'mediasfu-reactjs';

export function StandardRoom() {
  return <ModernMediasfuGeneric />;
}

Hybrid: use the room-owned modal lifecycle​

Pass the latest room parameters after each room-state publication. Opening and closing use that room's update function. Keep the component mounted for the room's lifetime so camera-off preview state and automatic restoration are preserved.

import type {ComponentProps} from 'react';
import {ModernBackgroundModal} from 'mediasfu-reactjs';

type BackgroundRoom = ComponentProps<typeof ModernBackgroundModal>['parameters'] & {
  isBackgroundModalVisible: boolean;
  updateIsBackgroundModalVisible: (visible: boolean) => void;
};

export function BackgroundPanel({parameters}: {parameters: BackgroundRoom}) {
  return <section aria-label="Camera background">
    <button onClick={() => parameters.updateIsBackgroundModalVisible(true)}>
      Choose background
    </button>
    <ModernBackgroundModal
      isVisible={parameters.isBackgroundModalVisible === true}
      onClose={() => parameters.updateIsBackgroundModalVisible(false)}
      parameters={parameters}
      renderMode="sidebar"
    />
  </section>;
}

React supports renderMode="modal", "sidebar", or "inline" on ModernBackgroundModal. These are presentation choices, not separate modal lifecycles. Do not add a competing local isOpen state or conditionally unmount the component whenever it is hidden.

With the camera off, the modal may acquire temporary preview media. Save stores the selection for a later camera start; it does not itself prove that video is being published. Let the modal release preview resources and let the room apply the saved selection when camera publication begins.

Fully headless: own controls, not transport internals​

The React helper requires a live camera for a published replacement. Serialize actions and show every returned error. Pass image for an image background; use the exported applyBackgroundBlur for a person-aware blur. Both use the SDK-owned processed-track lifecycle, rather than a second camera producer.

import {useState} from 'react';
import {
  applyVirtualBackground,
  applyBackgroundBlur,
  clearVirtualBackground,
  getCurrentParams,
  type HeadlessParameters,
} from 'mediasfu-reactjs';

export function BackgroundActions({parameters, image}: {
  parameters: HeadlessParameters;
  image: string;
}) {
  const [busy, setBusy] = useState(false);
  const [notice, setNotice] = useState('');

  async function change(mode: 'image' | 'blur' | 'remove') {
    if (busy) return;
    const live = getCurrentParams({parameters});
    if (mode !== 'remove' && !live.videoAlreadyOn) {
      setNotice('Turn on the camera before applying a background.');
      return;
    }
    setBusy(true);
    try {
      const result = mode === 'remove'
        ? await clearVirtualBackground({parameters: live})
        : mode === 'blur'
          ? await applyBackgroundBlur({parameters: live, publish: true})
          : await applyVirtualBackground({parameters: live, image, publish: true});
      setNotice(result.ok ? 'Background updated.' : result.error);
    } finally {
      setBusy(false);
    }
  }

  return <section>
    <button disabled={busy} onClick={() => void change('image')}>Use image</button>
    <button disabled={busy} onClick={() => void change('blur')}>Blur background</button>
    <button disabled={busy} onClick={() => void change('remove')}>Remove background</button>
    <p role="status">{notice}</p>
  </section>;
}

publish: true replaces the running video producer's track. publish: false is preview/local processing only. Install the optional @mediapipe/selfie_segmentation dependency and make its model assets available under your Content Security Policy when using React's background pipeline.

React's classic and modern modals, its headless helper, and Vue's background modal enable best-effort hidden-tab processing by default. Their keepProcessingWhenHidden option can disable it. Browsers may still throttle timers, capture, or rendering in hidden tabs; do not promise uninterrupted processed video. Check return-from-background behavior in every supported browser. Do not assume that a native SDK has this browser option.

Always render the processed local stream first​

MediaSFU's own prepopulateUserMedia and addVideosGrid logic uses this precedence for the self view:

const selfView = keepBackground && virtualStream
  ? virtualStream
  : localStreamVideo;

The safer React headless equivalent is getLocalVideoStream({parameters: latestParameters}), which also rejects an ended or disabled local video track. A custom local tile that always renders localStreamVideo shows the raw camera even though remote users receive the processed producer track. That mismatch is a rendering bug in the custom tile, not evidence that publication failed.

On every parameter or media publication, reattach the selected stream when its identity changes. Keep the self view muted. Never feed the processed output back into the segmentation input.

Removal and teardown​

  • Clear the helper-owned pipeline before switching cameras, stopping the camera, leaving, or replacing the room session.
  • Stop temporary preview tracks and animation loops that your application created.
  • Remove the saved selection only when the person explicitly chooses to clear it; camera-off alone may preserve it for the next publish.
  • Do not call the headless clear helper against a modal-owned pipeline.
  • Test camera initially off, preview, save, later camera start, apply, remote reception, local rendering, removal, camera restart, and leave.

Continue with device and translation availability by SDK, rendered media, and leave and cleanup.

Last updated on