Seamless HTML5 UI Prototyping

#1

… continued from Random Picture Thread.

Sssssweet. But why? :thinking:

I mean… do you have particular functions, toggles or widgets in mind we could try stuffing into this toolbar / sectioned pane layout?

Also – any thoughts on controller mappings? Right now I’m thinking about rigging button presses (not pointer related) as a secondary/aimless way to switch between panes.

Cool, but keep in mind this is just a quick proof-of-concept. My advice is to wait for and probably run with what @james_hifi suggests – in part because it’s likely to have a longer shelf life.

No – it seems only file, http, data, javascript and qrc URLs can be used with web views right now.

Unfortunately, adding atp support to QtWebEngine doesn’t seem very practical, but the next version of Qt makes it easier (so there is hope for the future).

There are some ways to solve the problem entirely on the content side, but the ones I know of just complicate things.

What I think would work best for now is “simply” adding a lightweight http-atp proxy to Interface.exe itself – able to answer ATP requests on mapped URLs like:

http://127.0.0.1:40180/{domain-uuid}/atp:/path/index.html

and hashed URLs like:

http://127.0.0.1:40180/atp:544c...433c/mime/type

Hmm, if you mean like jQuery, the stock qwebchannel.js include, stuff living under resources/ – then yes.

On the content side the image/nav is just vanilla HTML and jQuery. I literally unpacked its .zip file, edited the .html file to removing margins and padding (for a seamless display), and loaded into my “HTML5 UI toolbar” window.

The standard OverlayWebWindow would have worked too – except I didn’t want the window title bar or frame icons.

So made my own “HTML5 UI toolbar” window out of two files: a Client Script and the custom .qml it loads into an OverlayWindow.

Not entirely sure what you mean, but it sounds interesting!

What would the AC script do in that case (and would the Node.js instance be behind the firewall)?

#2

Why?
Well imagine if the location bar thing showed more than 3 places plus it gave events listings for the location

Not that we have anyone to visit or events to see
Or imagine the atp held images of its contents. Clothes shoes and stuff
Not that we can wear clothes and shoes and stuff

Aye it’s dumb just forget it

#3

Aha! I suspected you were sitting on some better ingredients… :wink:

The first part seems totally doable although people, places and things are pretty scarce still.

How about pooling upcoming listings into a single pane labeled “Events?” A Client script could fetch official ones from the Google Calender feed, and maybe unofficial ones too by scanning recent forum posts (for some kind of informal metatagging convention).

Also if we could convince @Menithal to incorporate metaverse-wide transponder beacons to Flares… then whenever you fired one it could automatically show up as a momentary “Events” pane item! :fireworks:

Hmm… like a Domain-specific kind of “Navigation Sitemap” – or maybe more like an “Inventory Catalog?”

How important would using ATP to host your images be?

#4

We use to be able to go to locations from photos ,could you hack the random pic thread and turn it into a bit of a HF tour? Unless this conrovenes HIFIs make it impossible to socialise policy

#5

Hmm. Can definitely extract a list of recent photos from the random pic thread, but not sure about accessing the raw images to get at the metadata.

Another possibility – I think a signal gets fired with the filename whenever a screenshot is taken. Maybe a Client script could take that with a plain text hifi:// URL and upload into an Imgur album, or something.

Well… even if such policies existed, I wouldn’t be too worried about them myself.

The Flare Gun + Transponder concept is an example of how people might connect and socialize even without user lists, identities and those kinds of policies. Launching a flare would disclose presence and location (presumably into a roster of recently detected flares); somebody noticing could jump to one of those locations. Identity never has to enter that equation.

#6

Cool!

I had a look but I’m struggling to understand how you are able to hide the chrome on an OverlayWindow with the toolbar you describe. Also the way you are toggling the visibility with the hud icon is confusing, is there another icon on top of the default of something? :clap:

#7

… there’s a lot of :crying_cat_face: involved still. Will try to get some code snippets up later as a reference point though.

Hopefully if some promising experiences and use cases can be found then maybe HiFi will consider adding an option to simply disable the chrome with an Overlay*Window boolean.

For now, here are the hacks I’ve been experimenting with:

  • Monkey-patching the live QML tree
  • in pseudo-code (nothing to do with jQuery, just summarizes well into those terms):

var chrome = jQuery(root).parent().parent();
chrome.find(“h1, img.icons”).hide();


 * QML "Window" eventually becomes available as `root.parent.parent`, but you can't access it from `Component.onCompleted` because it's not attached until moments later (see QML `Timer`).
 * `.findChildren().filter(function(x) { return /fingerprint/i.test(x) })` lets you "pan for gold" across descendants.  Find the chrome items and change their `.height`, `.visibility`, etc. to achieve desired effect.
 * This approach sucks... ugly, brittle and potential OS-sensitive.
 * HOWEVER -- only way found so far that can be deployed trivially as two files: .qml + .js

* **Modify the stock QML resources**
 * ie: experiment inside of `resources/qml/**/*.qml`

* **Reuse the stock QML resources** -- *rolling just your own replacement frame*
 * Simple in theory: create a thin `Window.Decoration` and hot-swap with `window.frame`
 * Unfortunately, virtually nothing under `resources/qml/**` can be imported beyond its tree.
 * Creating a second copy of `resources/qml/*.*` works... but I don't like that option.
 * Still exploring other possibilities to reuse without duplication.

Another thing I might try is no QML frame at all -- eliminating the whole dependency set and instead having the HTML side instrument any desired window dragging indirectly.

[quote="Triplelexx, post:6, topic:11285"]
toggling the visibility with the hud icon
[/quote]

So far I imagine this kind of toolbar making the most sense as something you'd summon on-demand (ie: not have up all the time).

Prototype was just a normal QML component -- fades in and out with the rest when you walk around and stop, toggles with the rest via HUD icon.  The chrome on OverlayWindows normally has a pushpin icon to vary that behavior for the window -- mine doesn't.
#8

Wow, that’s nuts, hats off for diligence! :clap:

I’ve noticed this as well, so I’ve been working within the resources folder.

#9

Phew! Was able to shed any uglier hacks and get the reuse approach working over the network (so now have a viable way to deploy this kind of solution).

For anyone who has struggled to use OverlayWebWindow, EventBridge and HTTPS together – one workaround seems to be hosting your own copy of qwebchannel.js.

Basically, instead of loading it using the built-in qrc:// url, place a copy next to your HTML file and include it normally (eg: <script src="qwebchannel.js"></script>). This allows for everything to arrive via https://, fixing the “mixed content” error messages and apparently allowing QWebChannel to do its thing.

#10

Hopefully if some promising experiences and use cases can be found then maybe HiFi will consider adding an option to simply disable the chrome with an Overlay*Window boolean.

That just sounds like a great idea generally. So in terms of process here – this is the sort of feature that you seem totally capable of implementing! And its something we definitely would pull upstream. If you don’t want to wait for one of us to get around to coding it, but would like the feature in the main branch sooner, you could open up a PR. I’m happy to help & review.

There is an understandable risk to spending time building something without knowing whether it is likely to get merged. One easy way to find out… just ask!

Option to hide browser chrome for HTML5 overlays === :slight_smile:

#11

Cool, but keep in mind this is just a quick proof-of-concept. My advice is to wait for and probably run with what @james_hifi suggests – in part because it’s likely to have a longer shelf life.

idk about waiting for me! :slight_smile: i think from a front-end perspective, what you have looks great.

i think the missing part is hooking it into the backend (interface!)

if you post your code here maybe i can adapt it so that when you click those location buttons, something actually happens in interface. what if i wrote a script so that it changed the skybox? or a certain zone’s skybox?

#12

Yeah definitely!

Right now I’m experimenting with some potential sugar that lets you define “shared methods” on one side and then call them from the other using auto-generated RPC stubs:

// Client script
Script.include('helper-TBD.js');

var toolbar = new helper.HTML5UI({
  source: 'index.html',
  chrome: false,
  width: 480,
  height: 240,
  shared: {
    setSkyboxURL: function(url) {
      return Entities.editEntity(SKYBOX_UUID, { /*...stuff...*/ });
    }
  }
});
...
function onSelectionChanged(entityIDs) {
      toolbar.rpc.highlightIDs(entityIDs);
}
...

<!-- HTML5 side -->
...
<script src="helper-TBD.js"></script>
<script>
var client = new helper.RPCBridge({
  shared: {
    highlightIDs: function(ids) {
      console.info('TODO: highlight DOM nodes related to ' + ids);
    }
  }
});
...
jQuery('.something').on('click', function() {
  client.rpc.setSkyboxURL('http://domain.tld/skybox.jpg', function(err, result) {
    if (err) console.error('error setting skybox: ' + err);
  });
});

Does this factoring seem like it would work for your known use cases too?

#13

… this is a somewhat tangential to Seamless HTML5 UIs, but if anyone is curious about the EventBridge adapter I’ve been working on my latest WIP is available as an enormous gist.

In context, a few relevant bits are:

  • Client script (can be loaded with Ctrl+Shift+o to see in action)
  • HTML5 content (normally loaded into an OverlayWebWindow, but can also be viewed from Chrome to inspect and test using mock objects)

http://imgur.com/JEbzZCs

(notice how the Camera.mode is kept in sync between the two sides – and how little code it takes in this approach to wire that sort of thing up)

Basically the EinsteinRosenBridge does most of the heavy lifting involved with rigging an EventBridge. And at that level you can use the same helper include, constructor and naming conventions – regardless of which side you are bridging from:

// Interface side
Script.include('EinsteinRosenBridge.js');

// create OverlayWebWindow:
var window = new OverlayWebWindow({
    title: 'Overlay Web Window',
    source: Script.resolvePath('index.html'),
    width: 480,
    height: 240
});

var web;

// setup the bridge:
var port = new EinsteinRosenBridge(window, {
    version: '0.0.0',
    key: 'interface-side',

    shared: {
        // allow Web side to set the Camera.mode:
        setCameraMode: Camera.setModeString,

        // and to query the current location:
        getCurrentLocation: function() { return Window.location; },

        // methods that the Web side has shared can be used too (after onload fires)  
        onClick: function(evt) {
            print('onClick!', JSON.stringify(evt));
            if (evt.id) // update background color of clicked button
                web.css('#'+evt.id, { backgroundColor: random_rgb() });
        }
    },

    onload: function(async) {
        print('shared methods (from Web side to Interface):', async.methods.join(' | '));
        web = async;
        // "push" example -- keep Web side updated with current and future Camera.modes
        web.modeUpdated(Camera.mode);
        Camera.modeUpdated.connect(web.modeUpdated);
    }
});

// HTML5 UI / Web side
// <script src='EinsteinRosenBridge.js'></script>
// <script src='jquery.min.js'></script>
var port, hifi;

jQuery(document).ready(function() {

    port = new EinsteinRosenBridge(window, {
        version: '0.0.0',
        key: 'web-side' + location.hash,

        shared: {
            // callback for receiving Camera.mode updates
            modeUpdated: function(mode) {
                jQuery('#cameraMode').text(mode);
            },
            // let Interface style elements by proxying jQuery.css()
            css: function(selector, k, v) {
                jQuery(selector).css(k, v);
            },
        },

        onload: function(async) {
            console.log('shared methods (from Interface side to Web):', async.methods.join(' | '));
            window.hifi = async;

            // forward button clicks to Interface
            jQuery('button')
                .on('click', function(evt) {
                    hifi.onClick({
                        id: evt.target.id,
                        x: evt.x,
                        y: evt.y,
                        button: evt.button
                    });
                });

            // retrieve current location from Interface
            hifi.getCurrentLocation(function(err, loc) {
                jQuery('#loc').text(err || loc.href);
            });

            jQuery('button.first-person')
                .on('click', function() { hifi.setCameraMode('first person'); });
        },
    });
});

Still complex, but also more concise I think.

#14

None of the buttons create a wormhole… :frowning:

#15

As an update to reference at the next JavaScript meetup, here’s how the “kitchen sink” version of my seamless prototype has panned out:

// from Client scripting side
Script.include('http://TBD.humbletim.server.tld/OverlayWebWindowEx.js');
var webWindow = new OverlayWebWindowEx({
    userAgent: 'Mozilla/5.0 Chrome/38.0 (HighFidelityInterface; OverlayWebWindowEx)',
    source: Script.resolvePath('index.html'),
    width: 360,
    height: 180,
    margin: '8px 50px',
    chrome: false,
    resizable: false,
    closable: false
});

I’ve probably wired-up too many options… but I like having options and here’s the current list.

(some of these are adaptations from the stock OverlayWebWindow, others involve deeper wiring into WebEngineView and some are just normal QML Item properties that are fun to play with from scripting – like rotation and scale)

// QML snippet
QtObject {
    property string url: "about:blank"
    property string userAgent: "(user agent)"
    property bool   chrome: true     // when true regular frame is used, when false seamless frame                                                                                                              
    property string margin: '4px'    // CSS-style margin descriptor 'TRBLpx', 'TBpx RLpx' or 'Tpx Rpx Bpx Lpx')                                                                                                     
    property var    icon: true       // whether to display the favicon (defined by web content) in full-frame title bar                                                                                                 
    property bool   pinned: false    // whether overlay always remains visible                                                                                                                                
    property bool   pinnable: true   // show the "pushpin" toggle symbol                                                                                                                       
    property bool   closable: true   // show the (x) symbol                                                                                                                                           
    property bool   cachable: false  // if cachable then (x) minimizes else (x) destroys                                                                                                            
    property bool   resizable: true  // whether the resize icon handle is available                                                                                                                                 

    property bool   animated: true   // whether changes to below properties get smooth-animated                                                                                                    
    property double scale: 1
    property double rotate: 0
    property double opacity: 1
    property double x
    property double y
    property double width
    property double height

    property string transformOrigin: '' // shorthand origin (eg: 'top left', 'bottom', 'center', 'bottom left', etc.)                                                                         
    property double z
    property var    progress: false  // webview normally manages this, but can also be scripted by providing a number
}

Most of those can be access later on the scripting side using custom read/write properties:

// Client scripting
webWindow.width    *= 2;    // double the window width
webWindow.chrome   = true;  // switch from seamless to framed mode
webWindow.progress += 10;   // add 10% progress to the mini bar 
webWindow.chrome   = false; // switch back to seamless mode
webWindow.margin   = '8px'; // switch to using 8px (seamless) margins

And on that side only a single “Script.include” is needed to boostrap things, although behind the scenes half-a-dozen other requests are still made as QML dependencies get resolved. I don’t have a good public link for that yet, but if anyone wants to help test sooner just send a DM.

The Open and Honest Content Provider Community Dialog Thread
#16

I’m away on holiday but will gladly test this with guidance in the middle of the week.