This repository is used for the purpose of storing and cataloging all of the synthetic video clips created for use in the SAIL-ON project's Video Activity Recognition Domain.
Each class/sub class has its own folder, and within each will be a text file detailing the naming convention, items used, in game location recorded at, etc so that they can be recreated.
The top level folders are the individual class names given to the video sets. WIthin each class folder, are folders for the game in which the videos were recorded. (Example: Swimming/GTA/clip.mp4). SOme classes may be firther sub-divided into differetn categories for convenience.
Also included in these folders are some extra files to help with figuring out how to recreate the settings used to record each set of clips.
In the misc folder are the character presets created for and used in recording the present clips in Skyrim and Fallout 4, which can be used in conjunction with the RaceMenu mods for each respective game, as well as a python script used to generate the annotations for these files, based on the annotation constraints file used for the domain.
For Skyrim, these preset files go into: "Skyrim Special Edition\Data\SKSE\Plugins\CharGen\Presets" For Fallout, they go into: "Fallout 4\Data\F4SE\Plugins\F4EE\Presets"
These instructions will provide a general guide for how these clips were recorded and the tools used to create them. For videos recorded using Elder Scrolls V: Skyrim, and Fallout 4, various user created game mods were used to assist in setup.
OBS (Download here) is a free streaming/screen recording software tool, which was used to capture the in game videos found here. See the Setup section below for how OBS was set up to record in game clips.
Steam (Download here) is a digital video game marketplace where the various games used to record can be purchased and installed.
The following games were used to perform actions in and record video clips. Grand Theft Auto 5 (Store Page) The Elder Scrolls V: Skyrim Special Edition (Store Page) Fallout 4 (Store Page)
In order to set up certain scenarios for recording, increase visual quality, and improve general game performance, many user created game add-ons (or 'mods') were used. In order to properly manage and keep track of these mods, a mod manager was used. Nexus is a website which hosts a large swath of these user created mods, so their manager was used. Download the manager here
Below are lists of urls for all of the various mods used throughout the process of recording clips in this repository. Mods were only used with Skyrim and Fallout 4. For all of these it is recommended to read their descriptions and usage guide's in order to understand what they do and how best to utilize them. For any new set of recordings, new functionality not present in the game may be desired, and therefore new mods may be necessary.
DISCLAIMER: These are user created and curated. We are not affiliated with or in contact with the mod authors showcased here. They have complete rights and control over the mod pages listed. Some of the mods in these lists contain optional content that would be considered 18+, or could be changed to incude 18+ content. Please keep that in mind as you navigate to these mods.
Skyrim:
ACC - Autorun Console Commands
Address Library for SKSE Plugins
BodySlide and Outfit Studio
Caliente's Beautiful Bodies Enhancer -CBBE-
Clean Save Collection - Main Quest Complete
ConsoleUtilSSE
Dynamic Animation Replacer
Dynamic Distant Objects LOD - DynDOLOD
Enhanced Lights and FX
Fores New Idles in Skyrim SE - FNIS SE
Footprints
Go to bed
Immersive HUD - iHUD Special Edition
KS Hairdos SSE
Majestic Mountains
MCM Helper
PapyrusUtil SE - Modders Scripting Utility Functions
Playable Monster Mod
Proper Length Arrows
RaceMenu
Realistic Ragdolls and Force SE
Realistic Water Two SE
Skyrim Script Extender (SKSE)
Skyrim Flora Overhaul SE
Skyrim SE Controller Interface Fix
SkyUI
Static Mesh Improvement Mod - SMIM
SmoothCam
Smooth Combat - Non Combat Animation System
Stronger Swimming Animation SE
Tempered Skins for Females CBBE
Tempered Skins for Males - Dressed Version
True Directional Movement - Modernized Third Person Gameplay
Unofficial Skyrim Special Edition Patch
Vivid Weathers - Definitive Edition
Woodchop Freecam
Wood Chopping To Max Carry Weight
Fallout 4:
AnimalArmors
Better Console - F4SE
BodySlide and Outfit Studio
Caliente's Beautiful Bodies Enhancer -CBBE-
Complete Vanilla Save
Custom Camera
Driveable Motorcycle Mod
Enhanced Lights and FX
Enhanced Vanilla Bodies-NeverNude Male
Fallout 4 Script Extender (F4SE)
Green Commonwealth - Performance Friendly Grass Mod
LooksMenu
Mod Configuration Menu
No Throwing Arc - Demolition Expert Perk
NSA No Settlement Attacks
PipboyRemover
Place Everywhere
Real Throwing Weapons
Realistic Jumping and Falling
Scrap Everything
See you sleep
Springtime Trees
Throwing Weapons In Fallout 4 (Bricks Included)
Throwing Weapons Westored
Unofficial Fallout 4 Patch
Water Enhanced - 4K Water
Once a game is installed, launch it normally through steam to access a small menu. Hit Options, and in the pop up, check Windowed Mode, then uncheck Borderless. Set Detail for the highest setting possible, and change Resolution to a smaller size to minimize video downscaling for recordings.
For setup instructions and how to use Vortex, as well as tutorials on how to install game mods, use this youtube video guide playlist
Once OBS is installed, there will be some key settings that need to be configured to match what was used during the recording process.
- Once launched, navigate to File > Settings > Output > Recording
a) Customize Recording Path
b) Recording Format: mp4
c) Rescale Output: unchecked
d) Rate Control: CBR
e) Bitrate: 500 Kbps
f) Preset: Max Quality - Navigate from Output to Video and customize the settings as follows:
a) Base (Canvas) Resolution: (Match this to the game screen resolution, while maintaining a 4:3 aspect ratio)
b) Output (Scaled) Resolution: 320x240
c) Downscale Filter: Lanczos (Sharpened scaling, 36 samples)
d) Integer FPS Value: 25 - Optionally, hotkeys can also be assigned to easily start/stop recording without having to navigate to OBS by navigating from Video to Hotkeys right below it and setting hotkeys for Start Recording and Stop Recording
- Now set up the actual game capture by doing the following:
a) Outside of settings, in the main window, go to the Soruces box
b) Click the + sign and select Game Capture
c) Create New and name apropriately
d) For Mode, select Capture Specific Window
e) Select game from Window list (game must be launched to be present)
f) Window Match Priority: Match title, otherwise find window of same executable
g) Uncheck all except Limit capture framerate
h) Hook Rate: Normal (recommended) - Also optionally, an output timer can be set so that OBS automatically stops recording a clip after a specified timeframe by going to Tools > Output Timer and customizing the "Stop recording after:" setting.
- Optionally mute sound in the recordings by hitting the Mute icons in the Audio Mixer section.
Once all of the required software is installed and properly set up, the process of determining what to record can begin.
Much of this process is dependant on the needs of the class being recorded for and the novelties being focused on. Some key recommendations though are as follows:
- Study the world map and find an ideal location.
- Plan ahead which character presets you will use, in game objects, actions, clothing, perspectives, etc.
- Create a filming set. Remember that recording in video games is more akin to live action. You as the player control the character, and need to act out the part.
- For some activities, using a game controller will give you more control over the character while recording
- Be sure to hide all traces of the games User Interface from the recording window as shown in the preview pane in OBS.
From here, the recording process for Skyrim and Fallout are different from that of GTA V, so they will require different instructions.
For some clips, you will want to use a free camera. Others may call for the third person perspective following the player. In order to access the fre camera, as well as other developer tools to modfy the world in-game, use the ~ key. From here you will have access to the games developer console. Some key commands include 'tfc' (Toggle Free Camera), 'tm' (Toggle Menu's. Note this also hides the developer console), tgm (Toggle God Mode), and 'showracemenu' (skyrim) or 'showlooksmenu' (fallout). For an exhuastive list of console commands and how to use them, use this guide for Skyrim and this guide for Fallout 4.
It is also recommended to take advantage of the quick save and quick load features as you make any changes to the games state in case the game breaks or becomes unstable as a result of using console commands.
As a last recommendation, use the character presets provided to quickly and easily swap between character models when recording sets of videos.
Finally, to actually record in these games, just simply line everything up, hit the start recording hotkey, and perform the activity in game. There are a lot of different approaches that can be taken to speed up the process such as grouping different types of novelties, together per character before switching to a new one. Changing characters will be the slowest part of the process (other than switching locations), so do as many recordings with a given character in a given location as needed before moving on ot the next one.
The process for GTA is a bit different in that there is a proper theater mode in this game that can be taken advantage of called the Rockstar Editor, as well as a director mode that lets you customize certain aspects of the game's world environment. In addition, while GTA does have a character selector, the options are more limited and it is harder to do things like change clothing, gear, etc. You can create more characters using GTA Online, but can only save up to 2 at a time.
Once you are ready and have your scene and character set up in Director Mode to record though, set up the in game recording, and then perform the action (or actions). The game will typically save up to 2 minutes worth or so, but you can also stop the recording earlier if desired. Once finished, exit from Director Mode and navigate to the Theater Mode, and select your latest clip. From here, you will have access to what is essentially a recoridng of not just the game screen, but the actual game world during the time recorded, which will allow for a large variety of alteration and customization in camera perspective. Make the most of each game recording, as switching between modes takes a significant amount of time.
To actually record the clips for usage, just like for Skyrim and Fallout, line everything up, press play on the video in game, and then press the start recording hotkey.
For a full guide on how to access and use these modes and the features they offer, use these youtube guides:
Director Mode Guide
Rockstar Editor Guide
Come up with a good, easy to sort through, naming convention for the video clips before hand. OBS will automatically name each clip with its timestamp. In order to prevent mix up and user error, name the videos as they are recorded, or in batches as they are recorded.
This effort was overseen by Adam Kaufman as part of PAR Government's contributions to the Sail-On Project under DARPA through the Video Activity Recognition domain.
Videos in this dataset were recorded by:
Adam Kaufman
Braeden Bowen
Brendon Grates
Connor Wyman
Ed Reznichenko
Kayla Cull