FrameNX

FrameNX is a simple unofficial desktop app for centralizing everything about on-PC mirroring and controlling of genuine Nintendo Switch consoles.

Essentially, this brings together screen and audio mirroring via (standard) capture cards, and sending of simulated gamepad inputs from gamepads that your console can't use directly. (The latter requires the sys-botbase homebrew, and thus a modded console).

This is mainly aimed at using a genuine Switch console as a cloud gaming host, with a desktop PC as a bridge running this program and a game streaming server application (eg. Sunshine). (There's no reason to use this just as a viewer if you don't need to send gamepad commands, there are much better video players out there.)

You will need to edit FrameNX-settings.json (generated after the program starts, please open it a first time then close it) with at least your own console's local IP address, under the nx_address key (port is already the sys-botbase's default). There are also other optional settings you can mess with, such as gamepad mappings.

Using the various actions in the program is possibly only via keyboard, no mouse, except for the virtual buttons overlay. Help for the keyboard commands is available inside the app by pressing the [P] or [H] keys. If a compatible (standard) gamepad is detected by the application, you can just trigger its inputs to have them sent to the console, in accordance with the configured mappings.

Known issues

• If the network connection with the console crashes, the program may think it's still sending any gamepad inputs fine, even if it actually isn't, and won't try to reconnect automatically. So, you will need to press [C] to force a reconnection.
• Setting the program to [f]ullscreen currently doesn't seem to use the correct display resolution on Linux (some desktop environments anyway). Please consider using the [W] option to get a borderless fullscreen window instead.
• Removal of [w]indow decorations and scaling window to full desktop size has some quirks, it works but not as intended; scaling only happens when restarting the program.
• Font size is fixed and doesn't scale to fit the window resolution. On very small horizontal resolutions (< 640px) some text will appear cut off, while on very large ones (4K?) it might be too small for some people. The default font_size setting is the best middle ground, but you can always change it.
• Due to a bug in the pygame.camera library on Linux, a video device crash or removal while it is active will cause a memory leak and waste RAM. A mitigation is implemented to minimize memory wastage in such cases, but, if during use you still end up wasting too much, please restart the program.
• It's impossible to change capture card settings (resolution and framerate) due to limitations in the pygame.camera API. I'm not sure how to tackle this, so you can only hope you get lucky and the default mode for your device is good enough for gaming.

Acknowledgments and Credits

• Python
• Pygame
• PyInstaller and staticx

Prior Art

• NXController

Licensing

Copyright (C) 2024, OctoSpacc

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Changelog

2024-06-01

• Fixed a coding issue that made the audio source not switch.
• Mitigated an issue on Linux where removal or crash of a video device could make the app leak memory infinitely.
• Made the keyboard keys used to trigger in-app actions customizable, and the keytips change accordingly.
• Made in-app actions also optionally usable via screen overlay buttons, and mouse clicks in the help menu.
• Right-clicking the screen can now toggle between normal operation and viewing the pause/help menu.

2024-05-09

• Fixed a regression in the code that broke the camera display.

2024-05-08

• Mitigate analog stick issue that made them drift after being released, as well as add additional user measures against bad sticks.
• Allow customizing deadzone and precision globally for sticks, or independently to cope with different sticks (eg. on the new 3DS).
• Introduced configurable sys-botbase setup commands. The default ones should mitigate previous input performance issues.
• Add virtual buttons overlay system, clickable via mouse and customizable in input, size, position, and visual aspects.
• Allow hiding/restoring all notice messages from view by pressing [T], or buttons overlay with [O].
• Add toggling of [w]indow modes, from bordered at original size to borderless fullscreen.
• Changed default config file name to FrameNX-settings.json to avoid conflicts with unrelated programs in shared directories. settings.json is still attempted to be loaded in case the new file isn't found.

2024-05-05

• Add some missing notice messages, and lines to the Help screen.
• Alert, and try to automatically (re)connect to console, when sending inputs raises exceptions.
• Show sent gamepad inputs on screen when debug mode is enabled.
• Show an error, but don't crash, if the audio system isn't working (eg. with root on Linux).
• Audio is now correctly muted while on the Pause screen.
• Add staticx to the build process for libc cross-compatibility on Linux.

2024-05-04

• First MVP version, added this README.
• Notices and error messages appear on screen at the same time and for just a few seconds.
• Cleaned the gamepad-handling code, fixed a few button bugs, allow working with multiple connected gamepads.
• User experience, performance, and stability improvements for the video rendering and event handling.
• Playback of audio input devices, with switching and disabling, and customizable backend flags.
• Allow to reconnect to the console's IP via a button in case of failure.
• Some UI improvements (keyboard buttons and strings changes).
• Log some extra information when debug setting is enabled.

2024-05-03

• First WIP version.
• Pause/Help screen.
• Viewing a video input device's feed, and switching or disabling them.
• Windowed or fullscreen mode.
• Video scaling smooth or fast, fit or stretched.
• Connecting to a console and sending inputs of a single gamepad.