D3D9, DXGI EndScene Hooking

Not sure where this really goes but I've stepped into adding something to FFXIVAPP (I'm the owner,dev,etc) and this community is the only one I know of that has the developers around that could do what I'm looking for.

I want to add overlay support for the plugins in the application and so far using SharpDX/EasyHook I've been able to rummage up some open source items to get at least FPS displayed.

Is there anyone who wants to contribute/help setup an overlay api to be used?

Ideally they could be Lua rendered items, or Xaml rendered items. I'm also going to state fully that all my stuff is open source, and anything going into it would have to be as well.

None of my plugins/app is coded to write anything to the game; just read.

Unless you're already doing a bunch of work in-process, actually rendering to the game's framebuffer is needlessly complex. Direct 3D doesn't give you any kind of GUI framework, you don't even get text (ID3DX can do rudimentary text, but it's not actually part of Direct 3D, has been deprecated, and the quality is quite poor). You basically have to do everything from scratch. In addition, you'll need another completely separate rendering pipelines to handle the new DX11 engine. I highly recommend avoiding this approach if at all possible.

You can achieve similar results by just creating a transparent window as a child of the game's window, and keeping its bounds matched to the game's client area. Using this approach you can do everything in pure C#, no P/Invoke, or hooking needed, and you get all of the power of WPF.

I was hoping to be able to show my widgets while the game is running full-screen. I've gotten a lot of requests to do that. I know another user got scaleform UI injected into the game with little slow-down but right now that's beyond my knowledge. I was looking at the hearthstone tracker for at least hooking the endscene which so far as at least worked in dx11/dx9 but after that you're right; so much work to get everything in the game.

Well, then you've got a long road ahead of you. Getting something to render is only the first step (which it sounds like you've managed to get at least partially working). You still need to hook the game's input (WndProc, at the very least, potentially more), so you can intercept and block mouse and key events as needed. Then of course you'll need to set up some kind of GUI framework.

For the GUI framework there are only a handful of options for open source projects. You can of course roll your own (which may be your only option if you need to stick to managed code). If you're willing to delve into C++, you have a handful of other options, such as CEGUI, and GWEN, but the documentation is generally sparse, and getting them set up can take some work. Of course, none of these options will let you use XAML.

To get XAML you're going to have to get WPF to render directly to the game's framebuffer, which is anything but trivial. I'm not even sure it's possible with DirectX 11. Getting it to work with DirectX 9, requires creating the device correctly when the game launches, so you would no longer have the option of launching after the game. You would need to inject and hook everything before the device is created when the game launches.

Hot damn...

I know someone told me they hooked all the stuff after the game was started and was able to inject scaleform but that person has since left XIV. I got rendering of text working correctly now with alt-tabs and such; it's just moving on to rendering widgets. You're right in that it's a long road. Currently when trying to render a saved png of a widget the alpha transparency doesn't work and once I inject that it goes to hell. ^^

FFXIV uses Scaleform for its UI already. No need to inject it, you just have to hook the right functions. That's still not going to get you XAML though; Scaleform uses standard Flash .swf files.

Hmmm, I could have sworn they are using Lua right now based on the sub-licenses. In any case I have DX11 hooked up and rendering text, and finishing up the stream interface. Should be able to render some stream data from a plugin.

Not sure on frame degradation yet; but game does't crash anymore.

Lua isn't a graphics library or UI toolkit, it's a scripting language. They use Scaleform for their UI, and Lua 5.1 for scripting.