Spediteur: A WYSIWYG Editor for SPIP

Hello all!

Since 2018 I’ve integrated several WYSIWYG editors with SPIP, for use in the frontend via the porte-plume / pencil plugin. Today I’m announcing Spediteur, an open source WYSIWYG editor that I plan to release for the SPIP platform.

The previous WYSIWYG implementations I’ve done have been quite useful for my clients and collaborators. As developer I can set up each page in HTML, and hand it over to the client/users who can then tweak the text and make simple changes to the website’s contents themselves, autonomously.

I love SPIP because of its stability, performance and elegant templating language. I like that whenever something is more complicated, I can write real code effectively. And I like that SPIP’s codebase is relatively small, so when I need to do something at a deeper level, I can read through it relatively quickly.

For many projects I have the need to make complex content editable for my clients and collaboraters, without them needing to know HTML. Spediteur allows me to create pages in full HTML in line with a project’s requirements, and then hand it over to the client. The client then sees and can directly edit the final result.

Before, the simple WYSIWYG implementations I’ve used over the years were never perfect, but recently the fact that there is no up to date WYSIWYG editor for SPIP at this point felt like a real limitation both for me and for the SPIP ecosystem. A good project to contribute to open source perhaps? Here we go.

Spediteur

Spediteur is a WYSIWYG editor for SPIP. It allows you to graphically edit your website.

It has the following features:

  • Inline editing: Built atop the porte-plume functionality, SPIPedit allows you to edit content directly in the side. No backend, no code: Just edit what you see, as you see it, on the website.
  • Full WYSIWYG HTML editing support
  • Typographical edits via keyboard shortcut and menu (bold, italics, underline, undo, redo)
  • Source view for editing raw html
  • Modeles support: SPIPedit highlights inserted modeles, so users know what they can edit, and what’s loaded as modele-component

It aims to have the following features in the future:

  • Upload images via SPIP documents
  • Improve saved HTML syntax (match as closely as possible to SPIP’s native syntax)
  • Bi directional source-graphical view updates (Updates in source view reflect in WYSIWYG view)
  • Hotload Modeles when they are added (Adding a modele loads it dynamically into the WYSIWYG view)
  • move and copy, paste modeles.
  • configure modeles (click on modele area, opens dialog to tweak modele source code, or for img modeles, SPIP document picker/upload
  • various improvements based on feedback and edge cases

A quick Demo of Spediteur:

Video: https://ursrig.com/IMG/mp4/spipedit_demo.mp4

Note: I noticed a project called SPIPedit already exists - I’ve therefore chosen Spediteur as the name of the plugin.


Technical points

When SPIP renders a website, it performs various syntax checks on the content of, say, an article. Users may have placed titles, links, pictures and modeles (SPIP custom elements) into the article. SPIP understands all this and then renders the page's HTML and the browser displays the page.

Spediteur is adding additional markup around modeles. With this additional markup, the WYSIWYG editor can show which areas are actually editable, and which are custom elements (modeles) that cannot be edited (they can be deleted, moved, or reconfigured, at least that’s the idea, see todo section above).

Using Contenteditable, Spediteur is able to show the website 100% as-is, so users can edit the site « live ». Squire editor is used under the hood to make the contenteditable experience solid.

On save, the HTML is parsed back into SPIP markup, meaning that the wrapping elements are removed and we have a clean, SPIP compatible output that is saved to the article, and can also be edited as usual in SPIP’s backend.

Next steps

At the moment, I just have a basic implementation that is usable for me, but needs polishing to become an actual SPIP plugin.

I’ll be using that to get feedback from users, to improve the system. Then I plan to package the solution as a SPIP plugin.

To make this possible, some changes may be needed to the SPIP core. I’ll be seeking out input from the community on how to make this possible.

I am interested to make an effort to build compatibility between Spediteur and SPIP’s core syntax and tackle the other todos. An example is that the current implementation adds

-tags around paragraphs, so in order to improve SPIP-compatibility, the parser would need to remove these consistently before saving. Newlines are also an issue. The goal would be that the conversion from source to Spediteur and back to source will create changes to the source just where edits are made, and not affect any newlines, etc to keep diffs minimal.

My questions

I have made some modifications to SPIP code and would like to know what's the best way to integrate this (Either by overloading the method in my plugin or by contributing to spip core).

1. controleurs/article_texte.html
Here I’m overloading a porte plume file, adding the WYSIWYG editor, which loads Squire JS, and custom JS to sanitize the WYSIWYG-HTML back into the porte-plume Textarea for saving (that also serves as fallback source editor). It also loads the edit bar. This is easy to put into a plugin.

2. spediteur_fonctions.php
Custom functions to convert source text to html text with Spediteur metadata.

3. ecrire/src/Texte/Collecteur/Modeles.php

Is there a way I need to either overload the function traiter_modeles() from a plugin, or make a modification in spip core Modeles.php.

The idea is to wrap each modele with some metadata, so the HTML editor can know both the modele’s metadata, e.g. <modele|id=1> and its output, e.g.

Modele 1 rendered
at the same time.

/**
<?php
	 * Traiter les modeles d'un texte
	 * @param string $texte
	 * @param array $options
	 *   bool|array $doublons
	 *   string $echap
	 *   ?Spip\Texte\CollecteurLiens $collecteurLiens
	 *   ?array $env
	 *   ?string $connect
	 * @return string
	 */
	public function traiter(string $texte, array $options) {
		if ($texte) {
			...
			$modeles = $this->collecter($texte, ['collecter_liens' => true]);
			if (!empty($modeles)) {
				include_spip('public/assembler');
				$wrap_embed_html = charger_fonction('wrap_embed_html', 'inc', true);

				$offset_pos = 0;
				foreach ($modeles as $m) {
					...
					else {
						...
						//Spediteur: add contenteditable wrapper.
 						//Wraps a modele with metadata needed for Spediteur editor
						if( is_spediteur() ){
						    $modele = "<div class='contenteditable_modele'  data-modele=" . json_encode(htmlspecialchars($m['raw'])) . ">" . $modele . "</div>";
						}

4. html/plugins/auto/crayons/v3.3.0/action/crayons_html.php
Lastly I would need to overload or adjust / remove any limitations to the largeur and hauteur max of the porte plume plugin:

public $largeurMaxi = 4000;|
public $hauteurMaxi = 4000;|

Thanks for reading! At this point 1 have only 3 questions:

  • Do you have any feedback, concerns, about my approach? Would you like the plugin?
  • About 3: ecrire/src/Texte/Collecteur/Modeles.php → Do you have any advise on this? Oveload, or modify in code?
  • About 4: html/plugins/auto/crayons/v3.3.0/action/crayons_html.php* → Should I overload this, or propose a PR to the plugin to increase? Should it become a Gobal variable e.g. define(_PORTE_PLUME_LARGEUR_MAXI, « 4000 »); ?

Thank you!
Urs

1 « J'aime »

Hello @ursrig very interesting project!

I didn’t had time for the moment to analyse this approach (I surely will) but just to let you know that long long old discussion Refaire un éditeur à base de CodeMirror + ajouts (#3720) · Issues · spip / porte-plume · GitLab which WAS at the very start about a WYSIWYMean editor, but then also about a WYSIWYG editor with many technical solutions compared and evaluated. And after a choice for ProseMirror/TipTap AND Markdown only, after months of development it resulted last year in this POC plugin : spip-contrib-extensions / Éditeur Markdown pour SPIP · GitLab

You can see 2 (old) screencasts in this thread : Refaire un éditeur à base de CodeMirror + ajouts (#3720) · Issues · spip / porte-plume · GitLab

To summarize:

  • markdown only (it’s a better sustainable core to pool and reuse existing components)
  • use of maintained libraries (TipTap / ProseMirror)
  • for the moment integration inside the SPIP admin, but it’s totally possible to integrate also in the public website
  • manage about all markdown feature
  • but also manage internal spip links (« article123 », « myobject456 »)
  • fully manage SPIP modeles in WYSIWYG : viewing, inserting and modification of existing, by using the YAML description of the modeles inspired by the plugin « Insérer modèles » (totally integrated inside the editor)

Obviously there is still plenty of work to finish, polish (and integrate in the public site too) :slight_smile:

1 « J'aime »

Yes, very interesting project.

And your editor works in the frontend ? great !
Also in the backend ?

For questions 1 and 2, I think you can overload in your plugin.

For question 3, could you publish a diff here, to understand what changes you need ?

For question 4, why do you need such big values ? :thinking:

BTW, the forge is open, on https://git.spip.net, you can create an account (and wait a little for its manual approval), and then you can fork spip/ecrire or spip/prive or any dist plugin, and make your own PR’s.

And to complete @rastapopoulos’s answer, some SPIP developers want to switch to Markdown syntax as the default in SPIP 5.
If this were to happen, a SPIP ↔ HTML plugin might be less useful…

But the decision has not yet been made officially, it is just being discussed.

Thank you @rastapopoulos and @nicod for your great comments!

Yes indeed the amazing porte-plume plugin that allows editing field’s contents in the frontend, has inspired me to make a WYSWYG editor for the frontent based on the same approach.

Adding a WYSIWYG layer in the frontend, users can edit the site directly. And see exactly what things look like.

Quick Screencast
I wasn’t able to embed the screencast video in my original post, it’s here: Spediteur: WYSIWYG Editor for SPIP - Urs Riggenbach

Markdown - Textwheel / Backend - Frontend
This will be a frontent-editor. The idea is to make the best WYSIWYG experience. And WYSIWYG is best when it’s in the frontend: When the container size matches the template’s, when all the CSS of the frontend is available, when the H1 tags and everything is rendered in the correct font, sizes, etc according to the actual CSS of the website.
(It could be used on the backend maybe and that’s something to consider in the future. It would require bringing the CSS into the backend, I’m not sure it’s wise).

About data storage: I have no opinion about markdown vs textwheel. As long as we can store modeles and also enter custom html when needed, I’m happy.

But for WYSWYG, whatever the backend storage format, something has to parse that source data, and create HTML for the editor. That HTML needs to wrap modeles and custom syntax with metadata so the editor can understand that these elements need to edited differently (e.g. open a dialog to configure a modele). All the rest can be edited normally in HTML, and upon save this HTML is parsed back into the original format (Textwheel or Markdown).

@rastapopoulos I will have a look at how you did that, no need to duplicate the works. My support for Textwheel is non-existant, I’ve just for now have modeles/documents working. Also no configuration dialog for that. So I will gladly take a look at your work.

* but also manage internal spip links (« article123 », « myobject456 »)
* fully manage SPIP modeles in WYSIWYG : viewing, inserting and modification of existing, by using the YAML description of the modeles inspired by the plugin « Insérer modèles » (totally integrated inside the editor)
*

@nicod I agree with 1 and 2, this I was able to do direclty in the plugin:

For questions 1 and 2, I think you can overload in your plugin.

Your other points:

For question 3, could you publish a diff here, to understand what changes you need ?

Please see the code block in my original post.

The simplest will be to pass an $option[« spediteur »] = true; into the existing function:

	public function traiter(string $texte, array $options) {

and add an if statement to wrap the html with the correct metadata (this would then be the pull request):

//Spediteur: add contenteditable wrapper.
//Wraps a modele with metadata needed for Spediteur editor
if( $options["spediteur"] ){
    $modele = "<div class='contenteditable_modele'  data-modele=" . json_encode(htmlspecialchars($m['raw'])) . ">" . $modele . "</div>";
}

For question 4, why do you need such big values ? :thinking:

The porte plume plugin has limited the max size of the editor to 700 px, but when editing a website where #TEXTE is wider than that, the editor is limited to that 700px max and it should be increased. I think there is no reason to limit the size, is there? I think 10000px would be fine, but it may be better to remove the limit: I don’t see what purpose it serves.

Have a great day,
Urs

1 « J'aime »

And WYSIWYG is best when it’s in the frontend

Sometimes it’s better, sometimes not :slight_smile:
If you have to switch from backend to frontend, to edit content, then back to backend to resume working on the structure (creating sections and articles), then maybe it’s not best, for example.

Personnally, I tend to include the critical css style from the main frontend content zone in the backend of SPIP, so that users can have a preview.

But for pure content editing, it is maybe best to do it in the frontend.
As always, maybe best, but not by default ?

But for WYSWYG, whatever the backend storage format, something has to parse that source data, and create HTML for the editor.

No, HTML is juste the format for the final rendering, in the browser. It is not necessarily THE pivot format between storage and editor, we don’t even need it at this stage.
I would add that HTML is just one of the formats that can be produced, among XML, JSON, PDF, etc. via API’s or action links.

But it’s a good point if you intend to support markdown, as it will certainly be (even if it’s not the defaut one) one of the supported formats for content storage/formatting in SPIP 5 (or later, but soon).

Please see the code block in my original post.

I’ve seen it, but it’s not easy to compare to the overloaded one and see what’s changed.
That’s the main goal of a diff/patch, to see the differences, line by line :slight_smile:
Are you familiar with git?
Basically, it involves creating a patch and copying/pasting it here.

The porte plume plugin has limited the max size of the editor to 700 px

Didn’t know that, and that sounds dinosaurish to me :sweat_smile:
You should make a PR on this, or at least open a ticket…

In any case, I can’t wait to try out your editor and put it to the test.

On your article, you say:

At the moment, I just have a basic implementation that is usable for me, but needs polishing to become an actual SPIP plugin.
I’ll be using that to get feedback from users, to improve the system. Then I plan to package the solution as a SPIP plugin.

The best way to have a good feedback is to publish your code on spip-contrib-extensions · GitLab
Or in your own private folder, but with public access and PR allowed, otherwise it’s closed source.

I’m just saying. :slight_smile:

And BTW, did you know there is a (very old) function that translates HTML to SPIP?

It’s called « sale » (dirty), and it’s a fitting name.

Thanks @nicod

I’ll send a PR for the porte-plume plugin to remove/adjust the max-size of 700px. I think we can just remove the max, but this can be discussed when I send the PR.

For the WYSIWYG, I do need HTML, as editing the actual HTML seems to be the best way to achieve 100% accurate representation of the source visualized in the browser.

Thank you for the link to the sale library! There are 3 ways to get the html back into the source format:

  • Source format in HTML, except modeles (simplest, what I have now, not good enough yet)
  • Source format in original Textwheel format (here is where sale comes in. Especially removing p tags seems to be a nightmare to roll on my own, so this library will hopefully save that for me. It will be great to see WYSIWYG editing with clean source code in the backend maintained.)
  • Source format in Markdown (I hope here I will be able to re-use an existing function, if you offer sites to be converted to markdown I would just re-use that to make this plugin compatible with that).

I will need a bit of time to proceed, will hopefully share something on the forge very soon. I’ll try to use sale as a way to get the html synthesized back into SPIP source code.

Best regards,
Urs

For the WYSIWYG, I do need HTML, as editing the actual HTML seems to be the best way to achieve 100% accurate representation of the source visualized in the browser.

That’s why all modern WYSIWIG libraries do not use HTML at all for editing, but always a pivot format (generally in JSON) with an abstract representation of all component and then it can be imported or exported in any other formats (markdown, html etc) for display or for save in database.

And that’s why for the other project I linked before, we chose Tiptap (itself based on ProseMirror), but also the very main argument: because we did not want to maintain our own WYSIWYG library (ultra complex on the long term), but take a widely used tool. :slight_smile:

True. Let me explore TipTap. If it has an inline editor that can be loaded in the frontend (e.g. fullwidth, with modeles rendered out), this would be an interesting choice.

It probably also comes with layout builders. The opinions differ on this, but I believe a bootstrap/bulma or similar layout build could be a game changer for users to be more autonomous when it comes to simple page layouts without needing the template layer.

This, combined with seeing exactly what’s produced « live » on the frontend… Could be very interesting.

I’ll have a look!