Integration of the GR Qt device driver

[jheinen]

Great package!

I would like to directly access the QQuickPaintedItem instance within the GR Qt driver. This would allow me to use native Qt drawing commands instead of displaying a pre-rendered image, which is not fast enough.

Fur this purpose, the GR Qt driver needs access to the Qt drawable (widget) to obtain the width / height and the vertical / horizontal resolutions (logicalDpiX/Y). That latter is not absolutely necessary.

Right now, we encode the address of a given widget / painter in our applications into a "connection identifier", which is then passed to the GR driver using an environment variable (GKSconid). Would be great to get similar information from QML.jl - probably as a string containing those QObject pointers separated by "!" (%p!%p).

The width() and height() methods seem to be available for the QQuickPaintedItem - I will see how to avoid the logicalDpiX/Y methods, which don't seem to be available.

[barche]

OK, so all is needed is a julia function returning the correct string? If so, this should be a breeze to add. Regarding the widget, it seems that QPainter has a device function that returns a QPaintDevice, which should have the required size and DPI info:
http://doc.qt.io/qt-5/qpainter.html#device

So I think we can get away with passing just the QPainter*, and the API will then be uniform between QML and regular Qt.

One thing that worries me is that the QPainter* is only accesible in the paint function here:
http://doc.qt.io/qt-5/qquickpainteditem.html#paint
Qt makes no guarantee that the painter will stay the same during the lifetime of the item it paints on, so theoretically this could be different each time paint is called. In practice, it seems to be the same pointer anyway, but it might be safer to pass a callback into GR that is called on each paint, and pass along the QPainter to GR that way?

[jheinen]

So I think we can get away with passing just the QPainter*, and the API will then be uniform between QML and regular Qt.

Sounds good - I'll check this.

... but it might be safer to pass a callback into GR that is called on each paint, and pass along the QPainter to GR that way?

The environment (GKSconid) is checked each time GR.updatews() is called - and that's triggered by paint. So we simply have to keep the environment variable "up-to-date".

[jheinen]

I made some tests and it turns out that we can obtain all necessary information using the device function:

painter->device()->width()
painter->device()->height()
painter->device()->logicalDpiX()
painter->device()->logicalDpiY())

So we just need the painter address.

[barche]

I have added a JuliaPaintedItem QML component that provides access to the QPainter pointer. See https://github.com/barche/QML.jl/blob/master/example/gr.jl for an example on how to set the environment variable. You need to checkout CxxWrap and QML for this to run. Normally you should be able to use this to build a real GR test?
I'm not sure how to format the pointer to a string without the Ptr{Void} part, though.

@tbreloff I wonder if this is generic enough to use in Plots.jl for any backend that supports QPainter as rendering backend?

[jheinen]

Thank you for the commit. I tried the following example:

ENV["QSG_RENDER_LOOP"] = "basic"

using CxxWrap # for safe_cfunction
using QML

using GR

qmlfile = joinpath(dirname(Base.source_path()), "qml", "gr.qml")

# Called from QQuickPaintedItem::paint with the QPainter as an argument
function paint(p)
  ENV["GKSwstype"] = 381
  ENV["GKSconid"] = "0x"hex(p)
  histogram(randn(10000))
  return
end

# Convert to cfunction, passing the painter as void*
paint_cfunction = safe_cfunction(paint, Void, (Ptr{Void},))

# paint_cfunction becomes a context property
@qmlapp qmlfile paint_cfunction
exec()

... but I get an error:

not converting unsupported field fptr of type Ptr
ERROR: LoadError: StackOverflowError:
in load_qml_app(::String, ::Array{Any,1}, ::Array{Any,1}) at ./:0
in include_from_node1(::String) at ./loading.jl:488
in include_from_node1(::String) at /usr/local/Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
in process_options(::Base.JLOptions) at ./client.jl:262
in _start() at ./client.jl:318
in _start() at /usr/local/Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
while loading /Users/jheinen/Home/Developer/GR.jl/examples/qml+gr.jl, in expression starting on line 22

Do you have any idea?

[barche]

Hmm, did you run Pkg.build("QML") after the checkouts? I forgot to mention that.

[jheinen]

@barche : Awesome. Now it works:

ENV["QSG_RENDER_LOOP"] = "basic"
using CxxWrap # for safe_cfunction
using QML

using GR

qmlfile = joinpath(dirname(Base.source_path()), "qml", "gr.qml")

# Called from QQuickPaintedItem::paint with the QPainter as an argument
function paint(p)
  ENV["GKSwstype"] = 381
  ENV["GKSconid"] = split(repr(p), "@")[2]

  histogram(randn(10000))
  return
end

# Convert to cfunction, passing the painter as void*
paint_cfunction = safe_cfunction(paint, Void, (Ptr{Void},))

# paint_cfunction becomes a context property
@qmlapp qmlfile paint_cfunction
exec()

Will provide a more challenging example in the GR example section soon ...

@tbreloff: GR goes interactive before our QtTerm and JSTerm is finished ...

[barche]

Allright, great news! I assume I need to checkout GR to try this?

Also, regarding interactivity, I think it would be nice to pass along mouse events from within the graph area (highlight a curve when the mouse is over it, allow selecting a curve, things like that). Would that be possible, and what would be required from the QML side?

[jheinen]

Yes - this only works with GR master. I'll check what's possible regarding interactivity (mouse move events, etc.) ...

[barche]

Just tried on OS X, but even after Pkg.checkout("GR") and Pkg.build("GR") I get:

GKS: Qt5 support not compiled in

[jheinen]

Sorry - I'll have to create a new pre-compiled binary for OSX first. I currently don't have access to the OSX build host - I'll be back in office on Tuesday and let you know ...

I am doing all these QML tests with a full GR installation (in /usr/local/gr).

[jheinen]

@barche : How can I obtain the width/height of the current drawable:

function paint(p)
  ENV["GKSwstype"] = 381
  ENV["GKSconid"] = split(repr(p), "@")[2]

  plt = gcf()
  w, h = ???  # need help here
  plt[:size] = (w, h)

  histogram(randn(10000))
  return
end

[barche]

I was just about to push an updated example, asking how to make the plot follow the window size, but you just answered that question. See the updated gr.jl example:

dev = device(p)
plt = gcf()
plt[:size] = (width(dev), height(dev))

The paint function argument p is now typed QPainter. I also exposed the logicalDpiX/Y functions. As usual checkout and build are needed first.

Now that my son relinquished the computer, I could test this on Arch using the gr-git AUR package, it is amazingly fast!

[barche]

(Continuing from #27 (comment), I think this is the more approptiate issue for this)
@jheinen Regarding the retina resolution, do the logicalDpiX and logicalDpiY functions not work?

[jheinen]

The logicalDpiX and logicalDpiY functions work, but they always return 72 (on macOS) both on Retina and non-Retina displays. Doesn't make sense for me. But I think it's not related to QML - will check this in pure C++ first ...

[s-celles]

I'm also facing (like @barche )

GKS: Qt5 support not compiled in

with example/gr.jl

Any idea?

[barche]

It seems the latest macOS GR binaries are not compiled with Qt5 support, I still get the same message. @jheinen can you confirm that?

[jheinen]

Yes - as we build GR from source we didn't notice that Qt5 is missing in the binary distribution.

I'm looking for a solution. It's somehow difficult, because our build system is (intentionally) based on macOS X 10.9. We probably need a special build here ...

[jheinen]

@barche , @scls19fr : Please checkout GR master and rebuild using ENV[GRDIR]=""; Pkg.build("GR"). This should download updated binaries for Darwin with support for Qt5.

If that fails - I linked the plugin against a Qt 5.7 (built from source) on a OS X Mavericks system - I have to think about another solution. Although I also have a ready-to-use macOS Sierra version, I would prefer to use one(!) version for all systems, which always worked fine (so far).

Which Qt5 distribution are you using?

[barche]

@jheinen I get the following error:

GKS: dlopen(/Users/bjanssens/.julia/v0.5/GR/src/../deps/gr/lib/qt5plugin.so, 1): Library not loaded: @rpath/QtWidgets.framework/Versions/5/QtWidgets
  Referenced from: /Users/bjanssens/.julia/v0.5/GR/deps/gr/lib/qt5plugin.so
  Reason: image not found

I am using the homebrew version of Qt (5.8.0 currently). Maybe a solution would be to submit a brew file to the homebrew project for GR?

[jheinen]

I rebuilt with the original qt.io Qt 5.8.0 distribution, but I still had to set the framework search path - in my case:

export DYLD_FRAMEWORK_PATH=/Users/jheinen/.julia/v0.5/Homebrew/deps/usr/Cellar/qt/5.8.0_2/Frameworks

This worked fine on macOS Sierra. We probably can automate this, e.g.:
joinpath(Pkg.dir("Homebrew"),"deps", ...)
But I don't know, how to obtain the Qt Version number.

[jheinen]

I found out, how to obtain the path information for the Qt Frameworks. But, the environment has to be set before starting Julia.

@barche : Do you have any idea how to proceed?

function initqt()
    try
        @eval import Homebrew
        if Pkg.installed("Homebrew") != nothing
            qt = Homebrew.prefix("qt")
            path = joinpath(qt, "Frameworks")
            if isdir(path)
                ENV["DYLD_FRAMEWORK_PATH"] = path
                println("Using Qt ", splitdir(qt)[end], " at ", qt)
            end
        end
    end
end

[barche]

I think it should be possible to set the rpath during the GR build. However, for O X and to ensure the GR and Qt binaries are compatible, having both available in homebrew would be the most elegant solution. Is the GR build hard to automate? If not, writing a script for homebrew should be easy, especially since homebrew probably has the dependencies require by GR? If GR is built by homebrew, the paths issue will be dealt with automatically.

[jheinen]

Please checkout GR.jl master and Pkg.build("GR") - this should set the proper symlinks for existing Qt5 installations (in Homebrew). During the build process, you will be informed about the Qt version.

[ufechner7]

Closing this because QT5 is not longer used.