Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mount state reporting #152

Closed
wants to merge 20 commits into from
Closed

Conversation

Dridus
Copy link

@Dridus Dridus commented May 22, 2017

As discussed in #110, there's no robust way currently to do something when an element goes into the document, as unready children in the same scope will prevent the whole scope from being installed. Often you can work around it using delay 0 =<< getPostBuild, but that would only work if the children became ready in the current frame and all ancestors of the current scope are already mounted in the document.

This PR adds a getMountState :: m (Dynamic t MountState) function by way of the MonadMountStatus class which ImmediateDomBuilderT implements. MountState starts at Mounting, goes to Mounted as soon as (but no earlier than) the time the corresponding DOM nodes get mounted in the document, and goes to Unmounted once they are removed.

This is implemented by creating a trigger event and holding it with a Dynamic at each scope, known as the local mount state, and zipping that with the parent scope's mount state. The root scope is handled specially by mainWidget, mainWidgetWithHead, and attachWidget', since it can never be Unmounted (as far as I know anyway :) ).

The majority of the changes are in ImmediateDomBuilderT, and while I was there I did several separate cleanups:

  • Replaced ((,,,) DOM.DocumentFragment DOM.Text (IORef (ChildReadyState k))) with a dedicated data structure Child
  • Added doc comments according to my (very new) understanding of how the pieces work together.

Caveat involving Unmounted: it looks like any performEvents inside a scope get suppressed after they get replaced with runWithReplace, so the natural code one might write to catch Unmounted might be like:

mountState <- getMountState
ffor (updated mountState) $ \ case
  Mounted -> do performEvent something
  Unmounted -> do performEvent something else

won't actually catch the unmount. Plumbing the dynamic out to the parent scope makes it possible to observe though, so I put a note on the doc comment. I'm not sure if there's a more elegant solution here.

I tested workingness of the MonadAdjust members for both regress by futzing around with https://github.com/ConferHealth/reflex-mount-test

If there are any scary edge cases that test doesn't help with, I didn't test them. I also didn't check performance, as I was unsure how best to go about that.

@ryantrinkle

Ross MacLeod added 7 commits May 20, 2017 00:09
…DomBuilder

also clean up some code around the immediate dom builder, and add some doc comments
compiles but needs testing
…s and fix the instance for ImmediateDomBuilderT

use the compiler, luke. *shakes fist at cabal*
also export MonadWidgetConstraints so it shows up in haddock
--
-- ___Note:___ once the current scope is replaced, any 'performEvent's in the scope will be cancelled and so if you want to observe the 'Unmounted' status
-- you have to plumb the mount state dynamic back out of the current scope so the parent scope can react to it.
class Monad m => MonadMountStatus t m | m -> t where
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if we can avoid using the word "Monad" in the name. I've been trying to move away from that (MonadAdjust, etc., ought to be renamed as well, I think, but that's for another time).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I liked it if only to distinguish between the type class which lets you fetch it and the MountState type. class MountStatus with data MountState could work though seems easily confusable. do you have a suggested name?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReadMountState?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetMountStatus to match the method it provides?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps HasMountStatus is a good fit; we also have, e.g., HasDocument.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like consistency. I'll go with that.

class DomBuilder t m => MountableDomBuilder t m where
type DomFragment m :: *
buildDomFragment :: m a -> m (DomFragment m, a)
mountDomFragment :: DomFragment m -> Event t (DomFragment m) -> m ()

-}

-- |'MonadMountStatus' represents an action that can be aware of whether the corresponding DOM built by the action is present within the document yet or not.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to call it a 'widget' here, instead of an 'action'? 'MonadAdjust' does kind of imply that any sort of action may be adjusted, and that's deeply connected to this, but I'm not quite clear on what it'd mean outside the contexts of widgets anyhow.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, if widget is the widely accepted term for "any unit of dom building"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected.

-- ^DOM structures are not yet installed in the document.
| Mounted
-- ^DOM structures are now in the document.
deriving (Eq, Ord, Show)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be any value added if we, and/or would it be possible to, add an "Unmounting" state?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had it originally and removed it because I wasn't sure how to wire it in... I think it'd need to go to Unmounting when runWithReplace is about to remove it but before the DOM updates happen?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think that would make sense. However, I'm OK with leaving it out for now to avoid adding more complexity to this PR.

@@ -56,7 +57,6 @@ module Reflex.Dom.Builder.Immediate
#ifndef USE_TEMPLATE_HASKELL
, phantom2
#endif
, drawChildUpdate
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional, because drawChildUpdate ought to actually be internal, right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, and the type(s) it deals in (ChildReadyState) are not exported

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, makes sense!

-- ^Action to take when the unready children all become ready, usually to install the document fragment.
, _immediateDomBuilderEnv_mountState :: Dynamic t MountState
-- ^'Dynamic' representing the current state of DOM nodes within the parent node with respect to the document as a whole. See 'MountState' for more.
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding docs and fixing the style! Much better!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -652,6 +684,7 @@ instance (Reflex t, MonadAdjust t m, MonadJSM m, MonadHold t m, MonadFix m, Prim
result <- runReaderT (unImmediateDomBuilderT f) $ initialEnv
{ _immediateDomBuilderEnv_unreadyChildren = unreadyChildren
, _immediateDomBuilderEnv_commitAction = myCommitAction
, _immediateDomBuilderEnv_mountState = zipDynWith min parentMountState firstLocalMountState
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@@ -112,6 +113,7 @@ type MonadWidgetConstraints t m =
, MonadSample t (Performable m)
, MonadReflexCreateTrigger t m
, PostBuild t m
, MonadMountStatus t m
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this should go in here. MonadWidget is a bit of an ad-hoc collection of stuff, which ideally I'd like to deprecate in favor of something that makes a bit more sense. We might, e.g., write a class Widget, which is explicitly designed to be beginner-friendly, and then move from there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my understanding of MonadWidget is that it's basically everything you can do while running live in a page, and corresponds to all the powers of Widget (from Reflex.Dom.Main). I can remove it, though I'm not sure what value that has until there's a better solution to all the powers.

can remove it if you want, though. it's trivial to add it as a requirement to use sites (and their callers)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I think that makes sense; we can keep MonadWidget going as a kind of "kitchen sink class" for the time being.

@@ -15,6 +15,8 @@ module Reflex.Dom.Main where

import Prelude hiding (concat, mapM, mapM_, sequence, sequence_)

import Reflex.Class (holdDyn)
import Reflex.Dom.Builder.Class (MountState(Mounted, Mounting))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whitespace style is just slightly different here. Running stylish-haskell from the root of this project should work - but in this particular case, it's just that MountState should get a space after it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah I've never used that tool so didn't notice. I'll fix

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected. I couldn't get stylish-haskell to run in a way convenient to check its complaints as it seems to want to be a pass-through filter only? running it from the root just had it hanging waiting for stdin.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I run stylish-haskell as stylish-haskell -i **/*.hs, after I've committed everything to git, and then I check the git diff. It's not ideal, but it works pretty well and makes it easy to revert.

liftIO $ writeIORef placeholders $! placeholders0
_ <- DMap.traverseWithKey (\_ (Compose (df, _, _, _)) -> Constant () <$ append df) children0
let result0 = DMap.map (_child_result . getCompose) children0
childInstallations0 = weakenDMapWith (_child_installation . getCompose) children0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think there may be a strictness issue here? Previously, we were casing on the (,,,), but now we're using a record selector. If my understanding is correct, this could potentially lead to the creation of a thunk that would still include the _child_result.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch, I think you're correct since the target map is a lazy one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected.

@@ -822,7 +913,7 @@ hoistTraverseWithKeyWithAdjust base mapPatch updateChildUnreadiness applyDomUpda
when (DMap.null newUnready) $ do
applyDomUpdate p
(children0, children') <- ImmediateDomBuilderT $ lift $ base (\k v -> drawChildUpdate initialEnv markChildReady $ f k v) dm0 dm'
let processChild k (Compose (_, _, sRef, _)) = ComposeMaybe <$> do
let processChild k (_child_readyState . getCompose -> sRef) = ComposeMaybe <$> do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only use of ViewPatterns in the file? Although I don't have a strong preference against ViewPatterns, I think in this particular case it might be just as good to do something like Compose (Child { _child_readyState = sRef })

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I had more uses but then backed off from them. I'll clean out this case to not have a one-off

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected.

@ryantrinkle
Copy link
Member

Overall, looks like a huge improvement, with just a few minor tweaks needed.

One big point remaining: what do we need to do to convince ourselves that this won't negatively impact performance?

@Dridus
Copy link
Author

Dridus commented May 23, 2017

okay, fixed all the low-hanging fruit. open issues I need some feedback on:

  • MonadMountStatus name
  • whether to include MonadMountStatus in MonadWidget
  • whether Unmounting is required and if so how to implement it

regarding the performance angle, it will undeniably add work to every DOM build though without a lot more digging I can't be sure how much. What work I know it adds:

  • newTriggerEvent at each MonadAdjust (transitive) usage and at the top level. Each of these it looks like implies two IORef creations.
  • triggering of those events each time a DOM addition or removal takes place are part of MonadAdjust, and at the top level. Triggering seems to be implemented in terms of adding something to a Chan (new MVar, take MV, putMV * 2) which then drops into hostPerformEventT and I lose track of its complexity around there
  • each time those events fire, at least Dynamic will depend on them (zipDynWith min parentMountState localMountState at each level). I have no idea what the performance impact of something upstream of a Dynamic updating is, either in the case where the Dynamic is used or unused.

@ryantrinkle
Copy link
Member

The perf impact seems a bit hard to estimate. It might make sense for you to try it out on a couple key use cases in your own codebase, and if it has an impact, we can extract that into benchmarks that hit those use cases.

@Dridus
Copy link
Author

Dridus commented May 23, 2017

I'll try it out on our two reflex projects (one which doesn't care about mount state and one which does) and report back if anything goes poorly. That said, neither is particularly performance heavy so may not be the best test.

@ryantrinkle
Copy link
Member

ryantrinkle commented May 23, 2017

As long as you have a couple of metrics that seem relevant, that'll be a good start! The top concern is that we can't create a regression in straight-line dom building speed (i.e. dom building without MonadAdjust in it).

@Dridus
Copy link
Author

Dridus commented May 23, 2017

I would be very surprised if it creates a regression there, as the only time it shows up is at the top level and in uses of MonadAdjust. In my testing using ConferHealth/reflex-mount-test I didn't see spurious mount events firing at the top level (or in MonadAdjust uses).

@3noch
Copy link
Member

3noch commented Jun 29, 2017

How's this coming along?

@Dridus
Copy link
Author

Dridus commented Jun 29, 2017

@3noch in my experience it works fine. @ryantrinkle is concerned about performance impacts though so will not be merging it into 0.5. If someone can provide some benchmarks that show its performance impact that would help move it along into mergetown. Unfortunately right now I'm underwater with work so I haven't gotten time to shepherd it further. https://github.com/ConferHealth/reflex-platform's confer branch is a reflex-platform with this branch of reflex-dom if you want to use it prior to being merged. The performance should be no worse for non-dynamic DOM building, but it definitely will impact dynamic DOM as it uses trigger events and dynamics to keep track of the mount state.

@Dridus
Copy link
Author

Dridus commented Jul 9, 2017

this branch had drifted w/r/t develop, and so I've merged develop into it and made sure it builds. there was one minor merge conflict around ChildReadyState, which I resolved

@Dridus
Copy link
Author

Dridus commented Oct 6, 2017

I've got the branch compiling against origin/develop, now working through testing it again. the upstream changes were pretty finicky as were the PR changes, so there may be some fallout I'll have to clean up

@Dridus
Copy link
Author

Dridus commented Oct 9, 2017

okay, tried out the basic tests at ConferHealth/reflex-mount-test using ConferHealth/reflex-platform@116fba2 which points at ConferHealth@c4dfbe3 and it seems to trivially work

@3noch
Copy link
Member

3noch commented Oct 20, 2017

Any more progress on this? I am likely going to need it soon.

ryantrinkle pushed a commit that referenced this pull request Nov 11, 2017
@ryantrinkle
Copy link
Member

Hey everyone,

@alexfmpe and I just got the full benchmark measurement suite to run, and unfortunately it looks like there's a bit of work to do here. Here are the results I got:

Benchmark Before After Ratio
01_run1k 641.023 742.253 1.1579194506281365
02_replace1k 416.629 430.351 1.0329357773942764
03_update10th1k 435.479 518.685 1.1910677667579836
04_select1k 33.381 32.797 0.9825050178245108
05_swap1k 11.457 19.265 1.6815047569171684
06_remove-one-1k 43.987 46.828 1.0645872644190328
07_create10k 3874.996 4759.231 1.2281899129702327
08_create1k-after10k 383.524 441.357 1.1507936921809327
09_clear10k 577.933 597.251 1.0334260199711731
21_ready-memory 10.384033203125 11.108634948730469 1.0697803764136082
22_run-memory 34.625877380371094 38.385414123535156 1.1085759272426492
23_update5-memory 40.12458038330078 45.36908721923828 1.1307055870949414
24_run5-memory 76.7357406616211 91.75919342041016 1.195781686985175
25_run-clear-memory 65.13986206054688 80.36016845703125 1.2336557971574647
30_startup 120.0 143.0 1.1916666666666667

It looks like perhaps we need to try to cut down on anything being done in the critical rendering path here. Since the benchmarks don't use mount state reporting, perhaps we can somehow avoid having them pay for it?

@3noch
Copy link
Member

3noch commented Nov 11, 2017

Would it be possible to have a combinator like elAttrDyn' that opts you in to mount-state reporting?

@ryantrinkle
Copy link
Member

@3noch Perhaps! It also might be possible to do it as a separate monad transformer. However, this code is pretty hair, so it's not obvious to me what the best way to go is.

@ryantrinkle
Copy link
Member

I should also note that I resolved conflicts and pushed the result here: https://github.com/reflex-frp/reflex-dom/tree/pr-152

@ryantrinkle
Copy link
Member

Oh yeah, also, the script for generating that table (given two "results" directories) is here: https://github.com/reflex-frp/reflex-platform/blob/develop/benchmarking/compareBenchmarkResults.hs

@3noch
Copy link
Member

3noch commented Nov 11, 2017

Another monad transformer would be fine. This seems like it gets deep into the spider/renderer though. Would that be possible without disrupting the rest of the code?

@Dridus
Copy link
Author

Dridus commented Nov 11, 2017

it doesn't get too close to spider, but it is pretty deep in the renderer. my thought for expediency if not most elegant was to add another hook or two to the ImmediateDomBuilderEnv, like commitHook, which gets called on mount and unmount, along with exposing some interfaces to tack another action onto the hook. then you could add the monad transformer to get you the dynamics by hooking

@ryantrinkle
Copy link
Member

ryantrinkle commented Nov 11, 2017 via email

@3noch
Copy link
Member

3noch commented Nov 11, 2017

@ryantrinkle Thanks so much for getting the ball rolling on those perf benchmarks, BTW.

@3noch
Copy link
Member

3noch commented Nov 14, 2017

@Dridus Could this be a start? #183

@Dridus
Copy link
Author

Dridus commented Nov 14, 2017

similar idea but no. that hook runs on each DOM update, regardless of what sort of DOM update it is. The thing the mount state reporting is trying to work with is that ImmediateDomBuilderT might wait indefinitely to put things in the DOM; it waits for all of its children to be ready. Usually that's not very long as it's only notReady for the first frame (e.g. dyn) but any widget can be notReady until it's replaced using MonadAdjust / Adjustable

@dfordivam
Copy link
Member

@ali-abrar @ryantrinkle

I did some analysis of the code with respect to the slowdown in BMs. There is about 25-30% slowdown in create10k. The result of other BMs are flaky, so we can ignore them.

I incrementally applied the changes of the PR and found that the slowdown is only due to creation of Dynamic values for each node with holdDyn, and then using zipDynWith. (The newTriggerEvent on its own does not introduce any overhead)

In the current design we are providing a Dynamic t MountState, and its value for a node depends on its parent node and own value. The parent node' value will be modified to Mounted state when all the child nodes are ready and the append action is committed. So there is flow of information from top to bottom

(This code is also capturing the Unmounted state, which will happen if the current node or parent is removed from DOM. As mentioned above the Mounted -> Unmounted transition cannot be used in the widget/scope itself. So if any cleanup action has to be performed it will have to be done by its parent widget.)

Possible solution / alternate design to avoid the slowdown:

Each node maintains a Dynamic / holdDyn, and the change in this is propagated to the entire sub-tree on commit action (either initial or adjusted). Every Dyn starts with Mounting and goes to Mounted.

I think that the user will be interested in mainly this Event / transition from Mounting -> Mounted. And this corresponds to the commit action of (some) parent node. So I think it should be possible to provide a similar functionality to user without maintaining Dynamic values in the code.

Now here the tricky part can be determining which commit action actually puts the child node in the DOM.
For the initial render the mountedEv will be the top / global event. For subsequent changes / adjustments, the commit action of the top node where the runWithReplace is initiated can be used as mountedEv for all its children.

@Dridus
Copy link
Author

Dridus commented Jan 4, 2019

@dfordivam thanks a lot for looking into this! I figured the Dynamic was probably the issue. I was imagining reducing the inbuilt support from the DomBuilder to just an Event for commit and Event for uninstall. I think the uninstall event is pretty important to allow external JS libraries to release resources associated with DOM, though the mounted state is definitely the more important. Once those Events were exposed the Dynamic could be built up from them using a monad transformer or something.

As far as when the mounting occurs, the patch should incorporate all the myriad places where that occurs. It's fairly complicated because it's when initial nodes get installed iff the parent is ready, or when the parent becomes ready and its parent is already ready, etc. I can help try to answer questions about the patch if you have them.


{-# INLINE hoistTraverseIntMapWithKeyWithAdjust #-}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This here is the culprit of the whole slowdown.

@dfordivam
Copy link
Member

So it turns out the whole slowdown was due to hoistTraverseIntMapWithKeyWithAdjust being marked INLINABLE instead of INLINE

I observed about 15% slowdown in the refactor PR #271 , and figured this out.

After fixing this the latest numbers are like this, we have about 2-3% change in the big 07_create10k
The only big difference now is the memory consumption.

Benchmark Before After Ratio
01_run1k 642.97 668.38 3.95%
02_replace1k 384.96 399.38 3.75%
03_update10th1k 548.88 559.83 2.00%
04_select1k 26.769 29.572 10.47%
05_swap1k 22.437 19.247 -14.22%
06_remove-one-1k 57.231 66.576 16.33%
07_create10k 4143.2 4263.9 2.91%
08_create1k-after10k 436.67 444.30 1.75%
09_clear10k 704.03 785.21 11.53%
21_ready-memory 9.1364 8.9039 -2.54%
22_run-memory 31.537 34.439 9.20%
23_update5-memory 36.420 40.381 10.87%
24_run5-memory 73.183 86.220 17.81%
25_run-clear-memory 61.905 74.906 21.00%

@dfordivam
Copy link
Member

Also my implementation of Event based approach in #272 has this result

Benchmark Before After Ratio
01_run1k 642.97 665.64 3.52%
02_replace1k 384.96 396.06 2.88%
03_update10th1k 548.88 530.50 -3.35%
04_select1k 26.769 27.318 2.05%
05_swap1k 22.437 20.141 -10.23%
06_remove-one-1k 57.231 63.706 11.31%
07_create10k 4143.2 4192.6 1.19%
08_create1k-after10k 436.67 446.20 2.18%
09_clear10k 704.03 745.64 5.91%
21_ready-memory 9.1364 8.8583 -3.04%
22_run-memory 31.537 32.003 1.48%
23_update5-memory 36.420 36.701 0.77%
24_run5-memory 73.183 74.235 1.44%
25_run-clear-memory 61.905 63.026 1.81%

@ali-abrar
Copy link
Member

Closing in favor of #273

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants