Project

General

Profile

Task #2394

master ticket for new documentation approach

Added by Mike Saunders about 4 years ago. Updated about 1 year ago.

Status:
In Progress
Priority:
Normal
Target version:
Team - Q1/2020
Start date:
Due date:
% Done:

0%


Description

GOALS

  • Bring all help content together
  • Use the XHP help as the primary source of information
  • Make it easier for end users to contribute fixes and new content
  • Retire old tools like the HelpAuthoring extension (buggy and generates bad XHP)

STEPS

  • Remove offline (F1 key) built-in help tool in LO 6.0 (Kendy is working on a patch for this)
  • For LO 6.0, create help packages (Olivier)
  • From LO 6.0 onwards, offline and online help will be displayed in the user's browser, for better rendering
  • Update XHP spec for additional sections, so that some can be used in online help, and others for books (Mike and Olivier)
  • For LO 6.1, implement an "Edit" button in the online help, along with login credentials and a user-friendly WYSIWYG editor that saves as XHP (Mike to work on this)
  • Then implement the workflow: use Git as ultimate source of XHP, and Gerrit for submissions of help content changes. When the user saves his/her changes, the editor creates a commit after submission, which goes to Gerrit for review by the docs team. (Maybe add a bot to do XML verification before integration?) (Mike to work on this)
  • Long-term goal for LO 6.2: extend XHP for merging with the books (eg sections or markup) - maybe even generate new guidebooks from that content. Add an editor preview mode - eg show how changes will look in the context of other help sections or a book
  • Another long-term goal: add "Search Ask" bar to online help interface
  • SEO
    • avoid double questions on Ask
    • categorize, indexes
      • current reference isn't usable -> re-tag, re-format
      • SEO not everything: is stuff online, should it be? tagging answers on AskBot, everything needs to be searchable and has proper metadata
      • Google is main entry point
      • "How do I do dots?" -> bullet list
    • entry pages
    • possibly linking from OOoForum

Files

editor_progress_1.png (305 KB) editor_progress_1.png Mike Saunders, 2017-12-07 16:26
editor_progress_2.png (282 KB) editor_progress_2.png Mike Saunders, 2017-12-07 16:26
editor_progress_3.png (614 KB) editor_progress_3.png Mike Saunders, 2017-12-07 16:26
Trumbowyg.png (242 KB) Trumbowyg.png Mike Saunders, 2018-05-18 14:38
Editor_10_July.png (161 KB) Editor_10_July.png Mike Saunders, 2018-07-10 15:10
xhp-editor-23aug.zip (1.21 MB) xhp-editor-23aug.zip Mike Saunders, 2018-08-23 15:25

Subtasks

Task #3470: Google Season of Documentation 2021ClosedOlivier Hallot

Actions
Task #3471: Manage publication of LibreOffice Guides 2021NewOlivier Hallot2021-12-31

Actions
Task #3472: Manage LibreOffice Help contents and toolsNewOlivier Hallot2021-12-31

Actions
Task #3473: Manage TDF E-learning on LibreOffice contentsNewOlivier Hallot2021-12-31

Actions

Related issues

Related to Documentation - Task #2938: XHP editorClosedOlivier Hallot

Actions
#3

Updated by Mike Saunders about 4 years ago

So here's an update on the XML editor situation. Buovjaga has helpfully pointed out a couple of options:

I'm investigating the latter now, as I believe WYSIWYG is important for encouraging people to work on the docs, without having to know what's going on with XML in the background.

Also waiting to hear from Jean Spiteri regarding his ideas, but haven't heard from him for a few weeks now.

#4

Updated by Mike Saunders almost 4 years ago

Still haven't heard back from Jean Spiteri, and ProseMirror turns out to be immensely complicated (the developers even describe it as a toolkit for creating editors, rather than an editor itself). So I've been working on an editing system that's built upon the new web help viewer that Olivier has created: https://helponline.libreoffice.org

First I added the "contenteditable" attribute to the "DisplayArea" div. This makes the div editable, but there's no immediate way to store the changes

So I added a form and a submit button. "DisplayArea" isn't part of a form, but I used some JavaScript so that when the Save button from the form is clicked, the "DisplayArea" content is copied into a hidden form field, which is then sent to a PHP script for processing. See how it works in the three screenshots attached to the ticket:

  • editor_progress_1.png - "DisplayArea" with "contenteditable" attribute, and a Save button which sends the data to process.php
  • editor_progress_2.png - The output of process.php, a one-line script which just shows the data passed to it. Here's where we have a limitation...
  • editor_progress_3.png - The data being passed (.innerHTML from the "DisplayArea" div) is the HTML after the stylesheet transformation, whereas we want the XML.

So that's where we stand right now. It may be possible to get the XML using different JavaScript, or we may need another approach.

#5

Updated by Florian Effenberger almost 4 years ago

  • Status changed from New to In Progress

Olivier Hallot and Mike Saunders - can you update this ticket? Ideally Olivier shares the state of play with Mike and outlines the next steps, and Mike updates the ticket as we've done initially. Then we can split individual larger tasks into subtickets.

#6

Updated by Mike Saunders over 3 years ago

Florian Effenberger wrote:

Olivier Hallot and Mike Saunders - can you update this ticket? Ideally Olivier shares the state of play with Mike and outlines the next steps, and Mike updates the ticket as we've done initially. Then we can split individual larger tasks into subtickets.

I'll arrange a catch-up call with Olivier now, and we can make a roadmap for the next steps. Then I'll update this.

#7

Updated by Jean Spiteri over 3 years ago

Just to let you know, I've finished a basic 1.0 version of my work if there is still interest. Mike Saunders and Olivier Hallot should have received a status update from me. If anyone wants to help, feel free to ask for updates on this ticket. Thank you Florian for letting me know (indirectly) about the mention feature.

#8

Updated by Mike Saunders over 3 years ago

  • Target version changed from Q1/2018 to Q3/2018

Olivier and I have had a catch-up call regarding documentation and especially the editor.

It turns out that Gerrit has an inline editor for files, which means that documentation contributors can directly modify XHP files and automatically generate patches in Gerrit itself, as Olivier describes: https://olivierhallot.blogspot.de/2018/04/direct-edition-of-help-files-in-master.html

So this is a big step towards the goal for LibreOffice 6.1 – making it easier and faster for (almost) anyone to update help content. Olivier is making a wiki page describing how to do it, and I will test it and update it where necessary.

The ultimate goal for LibreOffice 6.1 is full rich-text editing, which we will investigate once the Gerrit procedure is tried and tested.

(Also changing target version to Q3, as that's when LO 6.1 will be released.)

#10

Updated by Mike Saunders over 3 years ago

So a few updates from what I'm working on (the WYSIWYG help editor). I've investigated a few open source web-based editors and the most promising so far is Trumbowyg. See Trumbowyg.png -- this shows it editing the content from https://helponline.libreoffice.org/latest/en-GB/text/smath/guide/comment.html

The formatting is missing, but I'm trying to implement that. It's a bit tricky when paragraph tags in XHP can have so many different parameters. Also, the big issue will be getting the data back out in the right format after editing. I'll do more research.

Meanwhile I'm looking into XSLT-based options with Olivier.

#11

Updated by Jan Holesovsky over 3 years ago

Mike Saunders wrote:

The formatting is missing, but I'm trying to implement that. It's a bit tricky when paragraph tags in XHP can have so many different parameters. Also, the big issue will be getting the data back out in the right format after editing. I'll do more research.

Thanks for working on this! I've sent you some details in the mail, but just for the record here - I'm currently particularly excited about another editor, codemirror, see:

http://codemirror.net/demo/xmlcomplete.html

would be great to feed that one with the xhp XML schema & see if it is completing the syntax out of the box...

#12

Updated by Mike Saunders over 3 years ago

So here's Kendy's proposal of things to do/check before the conference:

At the current stage, to have something concrete before the conference,
I would suggest to:

* Deploy CodeMirror on some TDF VM so that we can see it all & hack on
  it

    + http://codemirror.net/doc/manual.html

* Add there the http://codemirror.net/doc/manual.html#addon_xml-hint
  and feed it with the XML schema of the XHP files

    + to get the autocompletion of the XHP syntax
    + if it is too hard to get it fully there, let's start with just
      a subset of the XHP

* Add there the XML syntax highlighting for XHP

    + initially the stock XML syntax highlighting
    + try extending it with one thing - like that all 'Heading 1's get
      a larger font (or something in bold is displayed in bold - or
      anything like that), so that we get a feel how hard it is to
      modify the syntax highlighting to our needs

* Evaluate 

    + is it an improvement (with the above implemented) compared to any
      other way of XHP creation?
    + will it be even better if we extend the syntax highlighting and
      autocompletion with more rules?
    + can we add some buttons that will automate stuff - like initial
      template of a new help page or so?

Would be great to version control all the above steps in git, so that
it is easy to see what changes were needed in the syntax highlighting
and in the autocompletion, etc.
#13

Updated by Jean Spiteri over 3 years ago

I can take care of these proposals as part of my proposal (if accepted) or separately (if it isn't).

#14

Updated by Mike Saunders over 3 years ago

The current state of the editor is shown in Editor_10_July.png. Key features:

  • General XML syntax highlighting
  • Autocompletion for some parts of our schema (see "version" for "helpdocument" – more will be added)
  • A button to reset the content back to a basic XHP template

Steps to take from here:

  • Complete the XHP schema for autocompletion
  • See if I can customise the CSS so that important bits (like headings) are more prominent, and less important things (like comments) are smaller
  • Deploy to a TDF VM for wider testing
  • Then start to implement the saving/Gerrit workflow
#15

Updated by Mike Saunders over 3 years ago

Quick update: I've added snippets and more XHP schema to the editor, along with undo/redo buttons and an option to create a new XPH document with a basic structure from scratch. I've shared a short video demonstrating the latest features with Olivier and Kendy.

Next steps: finalise snippets and schema, try to modify CSS as mentioned in previous update, then deploy to a TDF VM for wider testing.

#16

Updated by Mike Saunders over 3 years ago

The editor is now in the Git repo and Olivier is making updates:

https://gerrit.libreoffice.org/#/c/58500/

I'll catch up with it and work on more changes after the LO 6.1 release.

#17

Updated by Mike Saunders over 3 years ago

Latest version of editor attached (xhp-editor-23aug.zip) with the following changes:

  • Completed the XHP syntax (see autocomplete.js -- and note that entities and attributes with dashes need to be surrounded with quotes)
  • Fixed the problem with "meta" not closing properly, as you saw in the video a while back (for future reference, the problem was caused in mode/xml/xml.js: line 15: "autoSelfClosers", which had "meta" defined. If any other XHP entities have this problem, check there first)
  • Removed various unneeded CodeMirror files, to simplify the structure and hopefully make it easier for others to get involved
  • Some cosmetic cleanups
#18

Updated by Jan Holesovsky over 3 years ago

Mike Saunders wrote:

Latest version of editor attached (xhp-editor-23aug.zip) with the following changes:

Cool, thank you! Great stuff :-)

Olivier - can you please merge the changes with yours, and push to the dev-tools under Mike's name?

Mike - would be great if you start using the git repo too, so that you can send the stuff as patches, or push to gerrit directly :-)

#19

Updated by Mike Saunders over 3 years ago

Jan Holesovsky wrote:

Mike - would be great if you start using the git repo too, so that you can send the stuff as patches, or push to gerrit directly :-)

Yep, I'm still quite new to Git so I'll read up and use it in future -- just wanted to get these latest changes to Olivier ASAP :-)

#20

Updated by Florian Effenberger over 2 years ago

  • Description updated (diff)
  • Target version changed from Q3/2018 to Q2/2019
#21

Updated by Florian Effenberger over 2 years ago

  • Description updated (diff)
#22

Updated by Mike Saunders over 2 years ago

Just catching up on the editor: Olivier, did you hear anything from Guilhem about a test VM for Gerrit, to try out our new editor? He mentioned wanting to test it as a plugin first... If not, let's discuss it in the all-in call on Tuesday, when we can get the latest situation from Guilhem (I know he was busy with other infra stuff in the last couple of weeks).

#23

Updated by Florian Effenberger over 2 years ago

The plan is to have a call between Mike, Olivier and me next week, and then update this master ticket to the status quo
Mike to schedule the meeting, tentatively after the Tuesday team call

#24

Updated by Mike Saunders over 2 years ago

Notes from the team meeting in Munich:

New XHP editor - https://newdesign.libreoffice.org/help_editor/index.html

  • Need to change rendering in right-hand panel to PHP - to do it server-side (currently doesn't work in Safari/IE)
  • For "Open File" - picker/browser would be better
  • Buttons complete - make prettier
  • CodeMirror plugin can be built against our version of Gerrit - but JS bits didn't work https://vm178.documentfoundation.org/#/q/status:open - "Plugin "codemirror-editor" failed to load" - it's trying to load https://vm178.documentfoundation.org/plugins/codemirror-editor/static/codemirror_editor.html
  • Eventual plan to upgrade to Gerrit 2.16, which should make it work - upgrades will take a lot of work though, path is very problematic - so not before the conference; possibly before FOSDEM
  • For LibOCon: finalise interface and PHP rendering, and encourage doc editors to use it in copy-and-paste mode
#25

Updated by Florian Effenberger over 2 years ago

  • Target version changed from Q2/2019 to Q3/2019
#26

Updated by Florian Effenberger over 2 years ago

#27

Updated by Mike Saunders over 2 years ago

Quick notes from call with Olivier, Florian and Mike on 30 July

  • For XHP editor tasks from comment #24, Olivier thinks he can get most of them done for LibOCon Almeria (Mike can help where appropriate, and work on cosmetic improvements)
  • After that, identify what else we can deliver for FOSDEM
  • Look at converting more documentation into videos
  • Mike to make a SSO video (Redmine #2940) to make onboarding easier for new docs contributors
#28

Updated by Mike Saunders about 2 years ago

Olivier, Florian and I just had a quick meeting – Olivier will dedicate a chunk of time to finalise some parts of the new XHP editor, and Mike will work on the cosmetic side. For anything we can't implement, we will at least have the editor in a decent shape to show to the community (and hopefully get further help).

#29

Updated by Florian Effenberger about 2 years ago

  • Target version changed from Q3/2019 to Q4/2019

Mike, can you update the ticket according to what we discussed in the call, if there's anything to update in the description?

#30

Updated by Mike Saunders about 2 years ago

Florian Effenberger wrote:

Mike, can you update the ticket according to what we discussed in the call, if there's anything to update in the description?

What we discussed in the call was Olivier taking a dedicated week to work on the editor. The remaining steps from the description are:

  • For LO 6.1, implement an "Edit" button in the online help, along with login credentials and a user-friendly WYSIWYG editor that saves as XHP
  • Then implement the workflow: use Git as ultimate source of XHP, and Gerrit for submissions of help content changes. When the user saves his/her changes, the editor creates a commit after submission, which goes to Gerrit for review by the docs team. (Maybe add a bot to do XML verification before integration?)
  • Long-term goal for LO 6.2: extend XHP for merging with the books (eg sections or markup) - maybe even generate new guidebooks from that content. Add an editor preview mode - eg show how changes will look in the context of other help sections or a book

Of course, 6.1 and 6.2 have already been released, and we can't implement the second step until Gerrit is updated to at least 2.16. I don't know the status of the third point there -- Olivier?

#31

Updated by Florian Effenberger almost 2 years ago

  • Target version changed from Q4/2019 to Q1/2020

Olivier to update the ticket to reflect the status quo

#32

Updated by Olivier Hallot over 1 year ago

We had a new approach to the editor and is reflected in the ticket #2938.

The editor is in usable condition and is actually being used by the community.

https://newdesign.libreoffice.org/xhpeditor/index.php

#33

Updated by Olivier Hallot about 1 year ago

Latests 6 mo activities related to documentation

- restored extended tips from Help in the user interface, coodinating with translations reuse
- Dozens of fixes in the Help contents, catching up the gap between software development and proper documentation
- Several Improvements and fixes in the New Help framework
- Coordination of the documentation team meetings and resource allocations for them, discussions on technical issues for publications
- Management and execution of the Google Seasons of Doc 2020, got 2 projects, extended help on Calc functions and e-learning for Calc
- Staff activities
- Local community activities.

Also available in: Atom PDF