App design #5
Comments
|
It seems reasonable. You mentioned support for Ruby sample code. Remember, Markdown also support code. Enabling and disabling various GUI in YAML is very good. Let's consider styling (CSS alike) and also a Print button. A developer feature could allow enabling/disable preview mode. It would render the Markdown (or whatever backend) when enabled and show Markdown code when disabled. This could be useful for debugging Markdown mistakes, don't you think? Also, we may consider later to allow basic editing (non-preview mode would become edit). Take note that Kramdown provide a very good syntax highlighting system for major programming languages. We may consider in the future someone could write an interactive ebook for a completely different programming language, own sample code fully highlighted, ability to run non-ruby code, etc. |
|
Yes, kramdown has support for highlighting code. No markup for running code that I've seen yet. Also no markup for named sections (internel urls - not http:) which we would convert to (menu/side-bar). I agree that a style sheet mechanism is needed but it has to deal with what one tor two hackers can really do with Shoes/ruby - just 'implement css because every one knows it' is not possible although designing a DOM for a realizable Shoes-ebook would be valuable thing to do now, even more so if you envision .epub support. Perhaps we should be using our own XML mark up variation instead of interpreting kramdown/markdown. It make sense (and everybody knows and loves XML, right?) We'd need |
|
Now that I've looked a bit into to epub docs I don't a see a lot of future for this Shoes ebook brain cramp. No one will use it if it did work. |
|
Kramdown most likely has a way to tell us which language is highlighted. Modern syntax highlighting determine automatically the language. Then when we ask "Is it Ruby?", we can automatically add run code. YAML styling would be fine, no problem. We can let the user select fonts, font sizes, font colours, colours, background colour, etc. As long as we write Kramdown support as a backend, XML and EPUB are a tomorrow's problem. It would be good if you would explain why you don't see future for EPUB in issue #4 rather than here. |
|
Is terminology confusing us? Shoes (para, flow, stack, etc) is a kramdown backend as passengers kramdown001.rb demonstrates. I haven't found the documention for the callbacks yet so I can't say what we can or cannot do. Fortunately I have a local repo of the Shoes wiki to look at with it's menu/link structure to study. The menu pages and those Sadly, I have knowledge of of automating blogs moves from one engine to another (Manila to Typepad or Wordpress). hence the mvmanila.com domain name. Fixing Links/urls are a giant pain in the ass followed by dealing with images. Kramdown does support the gitub GFM version of markdown - the way to get syntax highlighting although it's not pretty for html output. There might command line options for that. There's also a huge bug in the GFM->HTML but I don't know if it applies to us or not since we don't have anything written that could test it. We also need to handle some other GFM syntax like Task lists and god help us, Tables
|
I am the author of kramdown001.rb.
It doesn't apply to us because we are writing a renderer. Our manual will be converted only once to Markdown and subsequently edited only in Markdown.
I believe convert_codeblock(el) is handling the rendering for the syntax, a.k.a. syntax highlighting. This will be triggered when a Markdown file finds a valid enclosure using back ticks. |
My error. Sorry.
The project is not to convert only Shoes manual. It's to convert markdown (in all its flavors into a Shoes value added (native app). |
|
I don't see a lot of value added in this ebook project now that I've been into the details. The manual, yes. Anything else -- not so easy. |
We should have a very simple first step. Anything else would be enhancement for future version. There is enough value in Shoes Manual as it is now, with few improvements (standalone, no more ftsearch, yaml configuration and styling, etc). This would make a very good first version. Additionally, I have been thinking for a long time about writing a small ebook on how to write a full Shoes software/game (i.e. Numinoes). Just a simple thing, nothing to sell on Amazon but would contribute to the community and perhaps bring more developers to use Shoes. So, no worries, we don't need no EPUB, XML, HTML, HTML5 to start with. |
|
Not a surprise, but krambown001.rb doesn't have all the responder methods that kramdown can call and also not surprising the api has changed, codeblock is now codespan. There are more and some of them may be difficult in Shoes (or not). I'm starting to understand the size of the task for just parsing GFM flavored markdown. There's a lot more involved thanconverting the crippled Shoes manual. |
You are correct that Kramdown001.rb doesn't have all responder methods. It is however incorrect about the API change according to the link you provided. Codeblock is for code and codespan is for verbatim.
Listen, you clearly see the task bigger than it actually is. The hardest parts were to come up with a working proof of concept using Picky, Kramdown and Rouge resulting into a promising rendering. I even filed various issues on Picky and gettalong/kramdown#214. All the foundations are laid down. Now it is about filling the missing responder methods and various styling adjustments to match Shoes Manual. There should not be any major work to do, nothing in the sense of what you did before like SVG or Chart API. |
|
Oh wait! You removed all the code from my original Kramdown001.rb in regard to Rouge. codespan is implemented to use Rouge. o_O |
I don't remember doing that. It's not something I would do to code I don't know. Comment out a line - yes, delete - no. |
It also comes as surprise to me. You do indeed usually comment out. The code is however present in the email I sent you. |
You haven't considered the problem of a directory of chapters, each with multiple markdown files (like the Shoes wiki has) that should save as one document (app) with links to to chapter/subsection that don't fire up a web browser if they are part of the "total" document. Oddly enough the syntax highlighter, proof of concept in the latest kramdown001.rb/kd_render.rb does not render anything. It does not throw an error, so I can get on with the bigger picture, |
* preprocess can download images * preprocess knows about about subdirs (or not) * much more cogitating needed to deal with internal links (#section in html) Github markdown doesn't seem to have an anchor-here. But we want them to create the ebook navigation menus on the sidebar.
|
Imagine if the Shoes manual was split into multiple markdown files(and directories) Hello ,Shoes, Slots, Elements, Furthermore, and Advanced. That would be the top left menu of a Shoes-manual.exe. If that app user clicks on Elements for example, we would expect that the sidebar would expand with links that the app handles itself without spawning a webrowser, like the proof-of-concept does. It get gritty when you learn that github markup doesn't seem to have a way to declare #here_is_an_anchor |
I would recommend a merging tool to obtain one single markdown file.
Anchors are not necessary. Shoes eBook should be able to build its own TOC based on what it cached from the markdown file(s). Same principle as the previous Shoes Manual implementation where it had a uri and a document for each search entry (for ftsearch, now Picky). Both a single markdown file or multiple markdown files should be fast enough to cache and build the TOC on startup. |
|
Yesterday's commit does capture the markdown header strings and level from a nested directory of them. And all the files names and downloads all the images. Before I get to rendering into something Shoes manual like, I need to that that more. I may need a DB to track some things. Leaning towards 'sdb' gem since we already use and Sqlite3 isn't all that fast at some things (and I don't like SQL, even less so in Ruby), but sdb is pretty limited. We'll see. I don't think one huge document is the way to go. There is a noticeable lag with rendering just the plot document and we aren't feeding the picky db yet or building the menu system (sidebar). Ah, github does have anchors - see here. ebook needs to handle them. |
|
Thank you for explaining. Sqlite3 would be ok but it seems a bit overkill. Ruby marshalling with or without YAML would be better. The best solution would most definitely be building the TOC in realtime.
Remember, Picky db only needs to be built once (or updated when markdown file(s) are updated). There is the lag you are talking about. It is subsequently fast to use, no lag at all. How slow is rendering the document in Shoes? We may consider generate Shoes/Ruby file instead of translating in realtime. This would also allow to generate the TOC as well as a sidebar.
Got it. A simple pattern matching should do here. |
|
We also need to exclude some .md files, and flag one as the landing page (chapter welcome , aka, preface) and order the .md files for that chapter by the author's preference instead of OS and Ruby file sorting defaults and save that in the yaml. An ebook would have a title, and a few more strings (author ) rendered over the cover.png, click to see the TOC. like Shoes spash screen does. The TOC has active navigation links to it's chapters. Each chapter has a landing page - a chapter preface and a subsection TOC over in the sidebar. Odds are some one will want the own background gradient or image for each chapter so the "builder" yaml should not exclude that possibility. |
|
Have you decided how to proceed with translation to Shoes? Markdown is compiled into Shoes app, we should consider to automatically update on newer timestamps to ease the edition of eBooks. From the README:
Why is that? We are currently supporting multi-platform deployment on Shoes. It comes to reason we should be able to support the same on Shoes eBook. May not be on version 1.0 but in the future somehow. |
|
I took a few days off from Shoes to install a raspberry pi3 (and compile ruby and shoes on it).
Because we are using merge_packaging - which doesn't install Shoes first, it installs a separate OS level application completely separate from what version of Shoes that might or might not be installed . Pain in the behind for Windows authors because they need NSIS installed but it's the only way to create something that works on Win 10 - unless someone figures out what MSFT needs for security settings and how and where to set them. On the plus side the author gets to create all their own icons and installer images and text and it truly is 'not a Shoes app'. Win 10 controls whats Shoes can do. Long requested feature of Windows and OSX folks who want to hide Shoes. There are gem issues - if you don't have them, you can't copy them and if you can't build them - you're sunk. We could provide a container (call it .shy ) and tell people to install Shoes, install this gempack, Then you can read the book in the .shy, but then you would need copies of all the native gems for all the platforms they care about authoring for. |
* try something hard like a copy of the Shoes wiki * download images in preprocess - not the end of this task * TODO comments highlight what need to been done in render phase. A whole f*cking lot and it might not be do-able.
No worries. Enjoy life.
We may suggest people to download NSIS. Otherwise we could move to Microsoft Windows Installer technology or integrate to Shoes eBook 7z which would allow to create a minimal self-extracting archive in command line (handled internally by Shoes eBook).
I am already doing this in Numinoes. I have developed a series of useful rake tasks that might be interesting for the community.
Indeed, native gems might be somehow challenging. Most authors are likely to use what's included and no more. We can do as many other Ruby libraries do, hook the require/include files, determine whether a gem is used or not, and copy it to the archive. |
We might as well include NSIS Unicode in Shoes eBook. NSIS Unicode is roughly 1.72 MB. Shoes is already 12.9 MB. Won't make a whole difference. |
|
Can NSIS install multiple apps (itself and a ebook.exe) - something to consider.
Only if its FOSS, and runs on Linux too. |
Obviously, that's the whole point of an installer. Numinoes has rake tasks to generate automatically a NSIS installer using YAML configuration file. It uses a NSIS file similar to the one I provided for Shoes as an ERB template. Integrating NSIS would be a good idea. So much that we should consider integrating it into Shoes as well.
You have in mind cross-compiling, isn't it? Windows Installer is standard and supported since Windows 95. It is very likely to run under Wine. This would be a suitable choice but it does involve rewriting the installer from scratch. We might stick to NSIS Unicode considering we already have something working relatively well. |
I meant - can a foobar.exe created by NSIS, install two apps in Windows, NSIS.exe and Shoes.exe (or ebook.exe) I build all 5 versions of Shoes once week with a script launched from cron - takes about 10 minutes to run and a third of that is for NSIS to do it's task. No need to launch a Virtual Machine and somehow tell windows to log me in, get devkit in the path and do, rake setup, rake, rake package, copy the file to some shared space on the VM host (and the log from console results from the build) I often go 2 or more weeks without having to fire up Windows. All of that could, possibly, be done with oraclebox's odd scripting stuff and poweshell, perhaps.
Last I looked, several years ago, Wine's emulation of win-threads did not work with Ruby and it was a vicious hang by Linux standards. Never bothered to try Wine on OSX but a sufficiently motivated person could do a mingw cross compile on OSX and port NSIS (if not done already). No Shoes user on Linux or OSX and often Windows, can (Shoes package an exe) that can install Shoes and then run their .rb app stored in a resource on Win 10. I suspect it is some sort of 'safe to install' mechanism that Win 10 implemented and who has the time and money to screw around with MSFT or APPL developer programs just to build a free app? |
There are two possibilities here: 1) telling in our NSIS script to require and install NSIS as a dependency or 2) include any executable files within Foobar and use it from its location (which means nobody will know NSIS is installed except Foobar). Shoes kinda takes a similar approach to the second where Ruby is included, with all Ruby own directory in Shoes, though here Ruby is also embedded into shoes.exe. For more information about the first approach, see embedding other installers.
This is very neat.
There would be no need to run Ruby on Wine but only Windows Installer. However, elaborating on NSIS gives us a good hint that we should continue with NSIS.
Not sure to fully understand the problem. Installers normally ask for elevated privileges to install an application. Other than that, I mentioned before we should get our own code signing certificate. It would also help to avoid false positive with antiviruses. |
How do I get one of these certificates? How much does it cost? Can I transfer it to someone else if I decide to not maintain Shoes in the future? |
|
Back to ebook and design. I think I've got the preprocess phase working (next commit) mostly working and and it creates clumsy yaml (instead of sdbm or sqlite3) to pass it to the renderer. A taste: ---
doc_home: "/home/ccoupe/Projects/shoes3.wiki"
sections:
shoes3.wiki:
:dir: shoes3.wiki
:title: shoes3.wiki
:files: []
images: {}
chapter-8:
:dir: chapter-8
:title: chapter-8
:files:
- Plot-Widget.md
- 8.0-New-Widgets.md
images:
https://cloud.githubusercontent.com/assets/222691/18075210/fdfecada-6e2f-1
1e6-8d62-3177925c789a.png: foo1.png
https://cloud.githubusercontent.com/assets/222691/18076413/f1120bba-6e3a-1
1e6-9e47-dc41ac41a3c0.png: foo2.png
https://cloud.githubusercontent.com/assets/222691/18076928/0f3a718c-6e3f-1
1e6-8c17-5fa53c840561.png: foo-3.png
https://cloud.githubusercontent.com/assets/222691/18192452/8afc0fd4-7091-1
1e6-8086-b4fdbe66a35a.png: plot-nubs.png
https://cloud.githubusercontent.com/assets/222691/18612528/827e11b0-7d19-1
1e6-9609-f041c99b1931.png: bar-graph.png
https://cloud.githubusercontent.com/assets/222691/18806307/21df17ec-81e6-1
1e6-908a-16c79f199b08.png: almost-good-scatter.png
https://cloud.githubusercontent.com/assets/222691/19026410/d3440eea-88e2-1
1e6-9e5a-4d338e4b2701.png: pie-finished.png
https://cloud.githubusercontent.com/assets/222691/20377785/07fb7878-ac4f-1
1e6-8af1-908c9d2b7953.png: radar-9.png
chapter-3:
:dir: chapter-3
:title: chapter-3
:files:
- Command-line-packager.md
- Polishing-Shoes.md
- Merge-Packaging.md
- Packaging.md
- Custom-Install-Scripts.md
- Gempacks.md
- Creating-Icons.md
- 3.0-Packaging.md
images:
https://github.com/ccoupe/shoes/blob/master/static/man-pkg-basic.png: man-
pkg-basic.pngThere are a lot missing convert_ methods that need to be written in the kd-render.rb some of which are quite arcane to me: convert_table, convert_html_element? Also syntax highlighting doesn't work . |
|
One other code signing issue and it's a doozey. Is the code signer legally liable for every packaged script someone else distributes. Shouldn't it be their certificate and not mine. |
This is good. Does the chapters need to be sorted in the YAML or is it Markdown business?
Certum offers a certification for Open Source projects at roughly 15$USD and some paperwork. It is yet unclear whether it is transferable.
There are some things you must do as you need to protect the certificate in a reasonable fashion, do your best for Shoes and its every third party included are free of virus and such, etc. If the certificate is compromised, you would have to request it to be revoked as soon as possible. Certum website states:
|
The order of the chapters is date order of the file on disk. It does need to be sorted but then the question is 'by what?' and then you realize no, they can't be sorted automatically - For instance there is an Appendix section which would sort to the top automatically and I probably wouldn't put it in the book either. Also the files in the chapter need manual placement - In this example, 8.0-New Widgets.md really should be deleted from the ebook but included in the wiki. Additional fun: That I've reopen Shoes issue 197 to discuss code signing. |
Somehow sorting shouldn't matter. The main wiki page is the entry point which will list other wiki pages as they see fit. Those other pages may list yet other pages. It comes back to TOC anyway.
Do you mean shoes/shoes3#196? Alright. |
A wiki TOC has few conventions. Chapter TOC as well, l are not marked as such, if they exist at all and for some reason they recommend links like Those conventions are just for github and not all markdown need to adhere to github rules. |
|
Home.md is the winner. Readme.md is for Github repository, for example, displayed on the main Shoes repository page. The Readme.md might be automatically listed in TOC in a furthemore (or alike) section but it shouldn't be listed by default. It's actually not relevant to a Wiki. |
Correct. For github wiki's. Home.md and what it points to is the way to build a a TOC for Shoes as the wiki author intended but that assumes gitub and well formed navigation at the source. FWIW, I've seen mulit-page documents rooted from Readme.md pointed into their repo doc dir or left loose in the project dir.- Not a wiki in sight. |
Got your point. Maybe we should tell Shoes eBook one entry point and it will take of the rest. So if one would like to build a Wiki, a Readme or any other kind of ebook, there you go. Interestingly, it could allow sharing content amongst books and yet easy to generate multiple books from the same source files. |
|
I was thinking the same. Once we know what the 'Navigation' home is then we can figure out what documents are menus/sidebars/section landing spots and where to put their content. Might require another pass through kramdown. |
* help_ebook.rb is what is used in place of shoes.rb in the packaged app (merge packaging) doen't work yet Because it needs a docment model in the yaml it loaded. (it does load the yaml) * begin creating the GUI to build/modify the document model in ebook.rb Just a brain cramp in case some one wants to follow along
* rename ebook.rb to be ebook-builder.rb * add GUI support to (re)order the the sections (aka chapters) * add GUI support to (re)order the files in display order in a section * Many things could be better GUI wise - go for it. * It's TOC that should work well enough to get help_ebook.rb tested * useless crud in yaml exists - probably * does not auto parse the nav document and build the TOC automagically because that's not a priority. Remember - we still can't render anything with acceptable fidelity.
|
Some observations as I try to understand the task. For Shoes ebook , kramdown render() returns an array of Shoes code(text) to be evaluated by Shoes to render a display. Basically .md ->fragments of Shoes/ruby once capture the book/section meta date (in the yaml) . Just an array of code for Shoes to eval. at the proper time and place. IF a goal is to have precomputed picky index files we might as well pre-compute and store the aggregate arrays of generated Shoes code. Those files would be platform independent, the picky indices, perhaps? That is what kramdown does. One file at a time, one-markup.file -> an other markup.file Granted, those files will be much larger than the md's but having both on them in Shoes/Ruby at one time seems a bit excessive. I'm just thinking out load, - add a compile step in ebook-builder that builds the Shoes snippets and indexes them. Include both (and any images) when building the exe/app/deb. The display code the user runs is pretty simple instead of the mixed up Manual only convoluted dewikify stuff you see in help.rb or help_picky.rb. |
* use @panel.clear {} for all tabs.
* declutter yaml, I hope.
* more Shoes-like GUI
* not complete it any world view. But, it's better.
* can also re-order TOC from the gui (in case you don't have the github navigation documents.) * still need ability to manually modify an auto created TOC. and detect md files not in the menus. * Should be good enough to start working on rendering.
|
Feeling better about the yaml and the possibility of rendering in a Shoes like skin with search. |
|
The progress is very satisfying. Keep it up. |
|
Without parse the documents, the Shell of a page looks like Shoes. Much more to do, but it is getting narrowed down into a manageable place where we'll learn all kinds of things. |
* help.rb and help_picky are deepely tied to Shoes manual * Random markdown can't be handled that way, * This commit just gets the title block nearly correct. Lots of works ahead.
I'm thinking about how and what the ebook app shoes do for a first time users (while keeping in mind the arcane Shoes manual. And that I want it all to be driven command line scripts, if desired. I see a couple of separate 'phases' and a .yaml
The user has some .md files and images and .rb sample code in a directory. That is his ebook. Our task is to provide a test viewer of their ebook and a way to convert their 'book' into a exe/deb/rpm/osx-tgz
ebook.rb (app) should put up opening screen/menus/buttons - create (yaml), view (yaml), save (yaml0 into their platform specific application. Obviously that
ebook.yamlif is file in their directory and not in where ebook was launched from.The text was updated successfully, but these errors were encountered: