What is your FFXI window resolution? If it's 1920x1080, you can use that as xRes and yRed to record the video. FPS should be above 10 if you don't want it to be choppy. If you have a good computer, you can do 30 easily. Quality should range from 1 to 100, usually there's not a whole lot of difference between 1 and 100 in the final video, so if your PC isn't quite as strong, I'd suggest going lower.
If it doesn't work it's usually because the resolution values are invalid. For example, I play on a 1920x1080 window (borderless fullscreen), and my PC is quite good, so I can handle the extra stress. Hence I type this:
//video start 1920 1080 30 100
Then again this to stop:
- AzureNight and Wictor like this