GPU-accelerated upscaling and frame generation for macOS
Features • Installation • Usage • Requirements • Building • License
MetalGoose captures a window with ScreenCaptureKit, runs MetalFX spatial upscaling and frame generation over the captured frames, and presents the result in a borderless overlay pinned to the source window. The GPU work runs on Metal 4 command queues, one each for the captures, MetalFX interpolation and presentation, so an interpolation never holds up a capture or the image a refresh is waiting for. It works on any window, not only games — anything that renders faster than it is being watched. Menus, popups, tooltips and dialogs the app opens over its window are captured with it while they are open, so they stay visible on the overlay.
A window is worked on at the resolution its content is drawn at. On a Retina display that is usually two pixels a point, but an app that is not drawn for Retina displays — a Windows game under Wine without its Retina mode, for one — draws at one, and is magnified to the display's pixels, bilinearly or as blocks of two by two. MetalGoose tells which from the first frames, by the relation each kind of magnification leaves between neighbouring pixels, and from then on takes every frame back to the content's own size before anything else: exactly for blocks (measured on such a game's frame) and to within one step of 255 for bilinear, and every stage after on a quarter of the pixels. With upscaling off the overlay is drawn at that resolution too, and macOS magnifies it to the display's pixels the way it magnified the app's own window, blocks as blocks: a quarter to two fifths less of the GPU for the same picture (measured), and the MG HUD drawn at that resolution with it. Native means the content's own resolution.
- MetalFX Spatial upscaling to the overlay's size. Scale Factor (1.0x–10.0x, or Fullscreen) sets the overlay size; Render Scale (Native, 75%, 67%, 50%, 33%) lowers the resolution ScreenCaptureKit delivers.
- Sharpening — Light, Balanced or Strong: contrast-adaptive sharpening (CAS) strength and anti-aliasing sensitivity.
MGFG-1 makes the images between two captures by interpolation. The Neural Engine does it where it can, and MetalFX on the GPU where it cannot, and a single choice is made for every capture so that the two work as one. The HUD's Frame Gen row says MGFG-1 and the multiplier; its Engine row names the engine making the images (and the size the Neural Engine works at, where that is not the capture's own), or says why none is: Starting while the Neural Engine's session is being built, or Unavailable where nothing can make the multiplier chosen just now (4x needs the Neural Engine).
- Neural Engine — VideoToolbox low-latency frame interpolation, for 2x and 4x. It leaves the GPU to the captured app: the GPU converts the capture to 4:2:0 once, and turns an image into colour only as it is shown. Above 1280×720's pixels it runs a slower way that is much closer to the real in-between frame (measured, 3 to 7 dB nearer than at 1280×720 or below), so a capture of up to that size is enlarged into it; a capture of more is given whole, up to the largest size it takes (1920 px on a side, 2.07 MP), and a larger one is shrunk for it, its images enlarged again as they are blended with the captures. Where the capture rate leaves no time for that, it works at 1280×720 or 960×540 in the fast way rather than giving way; so it does where the captures come unevenly, as a 60 fps source does on a 120 Hz panel (a tenth of them arrive 8 ms after the one before), since a pair that arrives while the last is still being made would go by without its images.
- Ready at once — the Neural Engine builds a model for every size it is used at, which takes a second or two the first time. The ten shapes Apple ships the network in, and the sizes used before, are built when MetalGoose launches, so a ladder with a prepared shape can start at it while the window's own model builds.
- MetalFX — interpolation with the media engine's motion field, for 2x where the Neural Engine cannot be used or cannot keep up. It is the more faithful of the two, makes the midpoint only, and uses GPU time.
- Multiplier — images presented per captured frame: 2x (the midpoint of each pair) or 4x (its quarters, on the Neural Engine), exactly as chosen, on any panel and at any capture rate. Where the panel shows fewer images than are made, each refresh shows the one nearest its moment; where no engine keeps up, the nearest one makes them all the same, and the pairs it has no time for go by without their images. The HUD's Target is never more than the panel's refresh rate.
- Interface and text stay as captured — where the content did not move between the two captures (an interface, text, a still background) they are blended back into the generated image, which keeps what did not move: the Neural Engine's images are lossy there, and MetalFX's gain a little. What counts is how much the captures differ compared with how much there is to differ, which is about how far the content moved, so a low contrast texture that is moving keeps the engine's image and is not cross-faded.
Interpolation holds the newest capture back by most of a capture interval plus the time the first generated image takes to make (some 40 to 50 ms at 30 captures a second). That delay is eased where it falls, so that a change of engine or load does not step the motion on the screen. The schedule runs on ScreenCaptureKit's presentation times — the compositor's, on the display's refresh grid — rather than on when each capture happened to reach the pipeline. For a game that presents in step with the display (30, 60 or 120 a second), the delay is also chosen so that the display's refreshes fall clear of the points where one image gives way to the next: every image is then shown on its refresh, where a delay set by the measured latency alone could put them on those points, and lose some of the images and pace the rest unevenly, at latencies that came in bands a refresh apart.
An engine is left at once when it stops keeping up, and not tried again for 30 seconds; a better one is taken only when it would keep up with room to spare and the choice has stood for 10 seconds, so a rate near a limit does not move the engine back and forth. With Render Scale below 100%, generation works on the reduced capture and the final scale-up treats captured and generated frames alike; only the Neural Engine takes the window's own size, when that fits it, which measured closer to the real image. Scene-cut detection avoids generating across hard cuts.
Resize and reappearance may redraw an existing image. The HUD counts new images when the drawable reports an actual presentation, so Generated + Passthrough = Presented remains true.
VRR (Display Settings) presents every image at the moment it is due instead of on the display's fixed refreshes. A panel that can vary its refresh then shows each image at its own time — the built-in ProMotion display does so on a 4.17 ms grid — so images whose rate does not divide the refresh rate are spaced evenly, and none is skipped. The system shows an image about a refresh after the time it was given, so each is handed over one refresh less ahead than on the fixed refresh and reaches the screen about as soon. Measured through the engine on the ProMotion display, three runs at each rate: at 80, 90 and 108 images a second the images reached the screen within 0.6 to 1.8 ms of their moment on average, against 2.1 to 2.6 ms on the fixed 120 Hz refresh, and at 54 within 1.7 to 1.8 ms against 2.3 to 2.5; at 120 about the same either way. They arrived at most 5 ms later than on the fixed refresh, and sooner at 54, 80 and 120. The presenter takes 0.2 to 0.5% of one CPU core more than the fixed refresh.
macOS varies the refresh only for a window that covers the whole display, so it takes effect with Scale Factor Fullscreen, and anything the window server draws above it puts the display back on its fixed refresh. So the HUD and, with VRR on, the pointer of Align Pointer are drawn into the picture itself: measured, VRR stayed on with both, where the HUD as a window of its own, or the pointer as a moving layer, turned it off. The pointer then moves with the images, and over a picture that is not changing it is drawn again once a refresh. In a window that does not cover the display the images are shown on the fixed refresh, as without VRR, and measured 21 ms sooner.
The display shows some images about 8 ms sooner after the time they were given than the rest, in stretches that come and go with or without the HUD and the pointer; while it does, the images are spaced less evenly (measured: a 95th percentile of 6 to 9 ms in such runs, against about 2 ms).
Post-process anti-aliasing that runs on the final captured image, with no need for depth buffers or motion vectors:
- FXAA — FXAA 3.11 Quality: relative edge threshold, a search along each edge to its ends, and a subpixel pass
- SMAA — Morphological AA: colour edge detection with local contrast adaptation, pattern-based edge blending and sharp-corner preservation
A HUD, drawn into the top-left corner of the overlay's picture, reports, live:
- Capture / Output / Generated frame rates, the Target output (capture rate × the multiplier in use, which Output is coloured against), and the panel's refresh rate
- Capture interval, total completed GPU command-buffer time per new presented image, GPU Budget (command-buffer duration divided by elapsed wall time), capture latency, presentation latency, end-to-end latency, and a frame-pacing score. GPU Budget includes capture, motion, interpolation, and render work; overlap and preemption can inflate it. It is a workload estimate, not a hardware utilization percentage. End-to-end latency uses the presented image's source compositor timestamp; it is not input-to-photon latency.
- VRAM, process memory, and CPU
- Cumulative counters: Captured, Presented, Generated, Passthrough, Dropped
| Component | Requirement |
|---|---|
| macOS | 27.0 or later |
| Chip | Apple Silicon |
| Xcode | 27 or later (macOS 27 SDK) |
| Metal | Metal 4.1: Metal 4 command queues and MetalFX, shaders built as Metal Shading Language 4.1 |
| Swift | 6.4 toolchain, Swift 6 language mode |
| RAM | 8 GB minimum, 16 GB recommended |
- Download the latest release from Releases
- Move
MetalGoose.appto/Applications - Open
Terminaland typexattr -dr com.apple.quarantine /Applications/MetalGoose.app - Grant Screen Recording and Accessibility before starting scaling
git clone https://github.com/Stallion77RepoOfficial/MetalGoose
cd MetalGoose
open MetalGoose.xcodeprojIn Xcode, use Product → Archive → Distribute App → Custom → Copy App.
Package the exported app with macOS's ditto, which preserves executable permissions:
set -e
release_app="/path/MetalGoose.app"
release_zip="$HOME/Downloads/MetalGoose.zip"
verification_dir="$(mktemp -d)"
trap 'rm -rf "$verification_dir"' EXIT
codesign --verify --deep --strict "$release_app"
ditto -c -k --sequesterRsrc --keepParent "$release_app" "$release_zip"
ditto -x -k "$release_zip" "$verification_dir"
test -x "$verification_dir/MetalGoose.app/Contents/MacOS/MetalGoose"
codesign --verify --deep --strict "$verification_dir/MetalGoose.app"Run these commands in a fresh shell. Upload the ZIP only after the extracted executable and code signature pass verification.
The project uses Apple Development signing to keep its identity stable across rebuilds. Select your own development team and certificate in Xcode, then build and run the MetalGoose scheme on My Mac. Screen Recording and Accessibility both require user permission before scaling. You can also select your team locally without changing the shared project:
xcodebuild -project MetalGoose.xcodeproj -scheme MetalGoose -configuration Release \
-derivedDataPath "$PWD/DerivedData" \
DEVELOPMENT_TEAM=YOUR_TEAM_ID CODE_SIGN_IDENTITY="Apple Development" buildTo build without a development certificate, choose Sign to Run Locally, or use:
xcodebuild -project MetalGoose.xcodeproj -scheme MetalGoose -configuration Release \
-derivedDataPath "$PWD/DerivedData" \
DEVELOPMENT_TEAM= CODE_SIGN_IDENTITY="-" buildMetal, MetalFX, and VideoToolbox use the same hardware and OS support with either signing method. Ad hoc rebuilds or a change of signing identity can require granting privacy permissions again.
Distributing a notarized app outside the Mac App Store requires Developer ID Application
signing, which is separate from Apple Development and local signing. Keep private keys,
.p12 exports, and account credentials out of the repository. A Team ID is an identifier,
not a signing credential. See Apple's code-signing certificates,
code identity and privacy permissions,
and Developer ID distribution documentation.
Use the same Derived Data directory for Xcode and command-line builds. In Xcode's
File → Project Settings, select a custom Derived Data location pointing to this
checkout's DerivedData directory. The commands above use that directory too; it is
already ignored by Git. Keep the checkout path and signing identity stable.
Open the intended bundle by its full path, for example:
open "$PWD/DerivedData/Build/Products/Release/MetalGoose.app"Avoid launching old builds in other Derived Data directories. macOS can register each
copy under the same bundle identifier, including during Xcode's RegisterWithLaunchServices
build step before the app has been launched, and reopen a different copy after a permission
change. If that happens, quit the copies, remove the obsolete build products, and open
the intended bundle by its full path. Changing the bundle identifier or signing identity
can require granting privacy permissions again. A fixed build path prevents extra copies
from accumulating; it does not replace macOS's permission checks.
CI builds and analyzes both Debug and Release on macOS 27 with Xcode 27 and the macOS 27 SDK.
English is the source and fallback language. The app follows macOS language preferences and
supports English, Turkish, German, Spanish, Russian, Simplified Chinese, Japanese, and Hungarian.
Translations include the settings, HUD, update messages, errors, and privacy descriptions.
User-facing text belongs in Localizable.xcstrings or InfoPlist.xcstrings; persisted setting
identifiers and error codes remain stable when the display language changes.
- Launch MetalGoose and grant Screen Recording and Accessibility access.
- Configure upscaling (MGUP-1), frame generation (MGFG-1), and anti-aliasing. Changes apply to a running session.
- Switch to the window you want to capture — it has to be frontmost, since MetalGoose targets whichever app is in front when scaling starts.
- Press
⌘⇧T, or return to MetalGoose and click Start Scaling.
Where the overlay is bigger than the window (a Scale Factor above 1.0x, or Fullscreen) the window is not where its picture is, so Align Pointer takes the pointer to the picture: the system pointer is kept inside the window and hidden, and the overlay draws one where the pointer appears in the scaled picture, so a click lands under it. While an app has taken the mouse for itself, to look around with, no pointer is drawn. Turn Align Pointer off to leave the system pointer alone; at 1.0x there is nothing to align. With VRR on, the pointer is drawn into each image rather than above it (see Variable Refresh).
| Shortcut | Action |
|---|---|
⌘ + ⇧ + T |
Start or stop scaling |
⌘ + ⇧ + C |
Show or hide the cursor sprite |
Both are global and outlive the main window, so closing it with ⌘W leaves the
overlay running and ⌘⇧T still stops it. While scaling runs, the main window is put
away; switching to MetalGoose (⌘Tab or the Dock) brings it back with Stop Scaling,
and the HUD is hidden whenever the overlay is.
One copy of MetalGoose runs at a time: the system hands a global shortcut to every app that registered it, so opening another copy (for example a build beside the installed app) quits the one already running.
All error codes are shown as an in-app alert.
- MG-UI-001: Frontmost app is MetalGoose; user must switch to target window.
- MG-UI-002: Target window not found for the selected app.
- MG-UI-004: No display found.
- MG-UI-005: Display ID not found for target screen.
- MG-UI-006: Display refresh rate unavailable for target screen.
- MG-UI-007: A global shortcut (
⌘⇧Tor⌘⇧C) is already registered by another app.
- MG-CAP-001: Target window not found by ScreenCaptureKit.
- MG-CAP-002: ScreenCaptureKit start error.
- MG-CAP-003: ScreenCaptureKit stop error.
- MG-CAP-004: Stream stopped with error.
- MG-CAP-005: Target entered macOS fullscreen — use windowed or borderless (windowed fullscreen) mode.
- MG-CAP-007: Capture reconfiguration failed when applying a new render scale.
- MG-ENG-001: Metal pipeline setup failed.
- MG-ENG-002: Metal device not available.
- MG-ENG-003: Metal command queue not available.
- MG-ENG-004: MetalFX Spatial Scaler creation failed.
- MG-ENG-005: Anti-aliasing pipeline unavailable.
- MG-ENG-007: CAS pipeline unavailable.
- MG-ENG-008: IOSurface texture creation failed.
- MG-ENG-010: MetalFX Frame Interpolator creation failed.
- MG-ENG-011: GPU command execution failed; the alert includes the stage and provider detail.
Codes are identifiers and are not renumbered when one is retired, so the lists have gaps.
This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.
Apple documentation this project was built against:
- Metal and compute passes
- MetalFX — spatial scaling and frame interpolation
- ScreenCaptureKit and capturing screen content in macOS
- MTLTexture and CVPixelBuffer — the IOSurface-backed path between capture and render
- CAMetalDisplayLink — the frame clock the render thread runs on
- VideoToolbox — motion estimation on the media engine and low-latency frame interpolation on the Neural Engine
- AppKit — the overlay window