we are a big partisan to start a new page provider using XML as storage.

I know the objections, but... I do not care.

I think the benefits outweigh by far the drawbacks.

I am currently working on it, as a derivative of FileSystemProvider.

Some of the potential benefits:

  • versionning will become much easier to deal with, using its own mechanism, where each file as a unique file name (# from Wikiname) and versions of a same page are just linked together.
  • properties are also versionned and stored with a version of a page,
  • structured extensions with the possiblity to have a a set of pages acting like a semi-structured database
  • categories becomes a tag
  • an authorizations and authentication implementation will be much cleaner
    • structured documents for users and groups, and authorizations,
    • matching with a tag of a document
  • keep wiki text editing within a tag, or replace with html/xhtml with WysiWyg editor
  • possibility to add abitrary tags to any pages
  • structured search and indexing (ie Lucene +XML extensions) or Xquery engine
  • possibility to get a better integration (dynamic) with other tools (RSSfeeds, Touchgraph, ...)
  • and so on...

To me it is the next natural evolution of a wiki. I know only a few that use that :

But the implemtation is fairly crude or the projects are dead.

I will post the code here when it is available in the next couple weeks.

Let's see what JSPwiki users respond.

Any feedback?

-- PhilippeO

I would like to list some of the drawbacks as well, just so that we keep both sides into account:

  • XML wastes space, being very verbose
-- But storage is cheap, and becoming dirt cheap. I can buy 1Gb for about 1$/1E. I have 120 Gb on my Dell laptop. I cannot see the end of it. 10 years from now, I may have like 1 petabyte on my desk. Why not make good use of it? -- PhilippeO
  • Doing diff-based storage (like with RCSFileProvider) is not so efficient anymore.
-- The stance to take here is not to make a diff-based efficient storage, but to use space that is cheap, in a way that makes sense for the application -- PhilippeO
  • XML parsing/storing adds an extra layer of complexity and a possible break point.
-- Well you are already parsing WikiMarkup, and also property pages, and so on... What is complex and fragile about it? No more fragile and complex than using proven well tested XML parsers and processors like Xerces, XT, and Dom4J or JdoM. It would actually provide a more uniform way of parsing anything that needs parsing in the wiki. Why would that be a break point? -- PhilippeO
  • XML parsing is a significant overhead.
-- I agree with the overhead, but also, processing is cheap : I have a 1 year old 1.2Ghz P3M laptop, which is already an antique by today's standards. I cannot see the end of it for pretty much everything except very large build tasks. Speed I get with surfing, email, word processing, code editing... is quite to my liking. Let's put the superfluous Gigahertz to good use! -- PhilippeO
  • Using XML as storage means that you no longer can edit the page contents as easily off the disk
-- I have never edited pages off the disk. If you manage any kind of wiki with more than one user, this is bad practice anyway. The purpose of the wiki is to offer web based collaborative editing. I will forego this feature without regrets -- PhilippeO
  • If you use XML also to denote WikiMarkup, then your users will make invalid XML :-).
-- The idea here is to have editing helped by the application : either with a wysiwyg editor, or a jsp to construct an edit form based on the document's tag structure. You could also add arbitrary tag, whicle editing (akin to adding a form field). In that mode, you actually never touch any XML as a user. You just browse adn edit forms. In some case, you have only one field, and the content is WikiMarkup. In other cases, there may be many more fields that describe a nice business document. But you cannot create invalid, not well formed or invalid XML. You could also parse the WikiMarkup in XML, and "unparse" the XML in WikiMarkup for editing, or unparse it in another representation (RTF, PDF, ...) . -- PhilippeO

Please list more, these are just off the top of my head.

But having said all that, I still think this is probably a good idea. I wouldn't go as far as putting in stuff like categories as tags, simply because currently there is no notion of a category within the Wiki code itself, and you would have to do AI-level parsing to figure out which pages are categories and which are not =). (No, using *Category does not help, simply because there are other people than English-speaking using JSPWiki).

-- Hence the value of using tags : a category tag could have synonyms tags in other languages, or values in different languages. Also talking about language, I could have the same page, one tagged as being in English, one tagged as being in French, or Chinese (well French 1st, that will be more usefull to me at least (:-)... -- PhilippeO

I don't also think that it helps authorization/authentication one bit, simply because the decisions are made based on parsed structures anyway - the WikiEngine should not know how the pages are stored, and should not care.

-- Well at least for this 1st version, this would be incompatible with existing WikiPages. The XML stored pages could not be mixed with FS or RCS based pages. The Wiki engine would not know, but the authorization could become much more clean to deal with : one model for stored structures, not one for properties, one for pages, one for versions, one for authorizations. -- PhilippeO

It would also probably require a separate editor as opposed to the current model of in-line tags. -- Just a different EditPage, using a mode very similar to WidgetWeb -- PhilippeO .

But storing the WikiMarkup as-is inside an XML tag should work - this way the entire page with all its versions would be a single XML document, which may be useful in certain cases.

-- There is already an implementation of XmlStorage where all versions of a page are stored in one Xml document : DarachWiki Another implementation of XmlStorage is Chiki. I like neither of them. They use a fixed schema, so you cannot add new tags to a page. And I would not want to store all the version of a page in one document, but rather use one version = one file, with unique file names (using a SHA1 hash for the name on the file content). Files names and files become therefore globally unique, and you can make them write once then read only. Versioning needs to be resolved by linking : a version has a link to an older version of the same object. This mechanism (derived from Oceanstore) could open the road to a distributed wiki, a kind of P2P wiki, where pages can be produced somewhere on some server, and synchronized, replicated to other servers. And the server could be a desktop (may be a StandaloneWiki part of a federation of other StandaloneWiki) That is also part of my next plans for my JSPwiki... -- PhilippeO

But be my guest, do make a contribution and post it here so that people can try it out. Just don't forget to observe the coding standard :-). -- JanneJalkanen

-- Thanks for the encouragements. I carefully selected JspWiki to try that after reviewing in details all the Java based wiki out there. The only other serious contentders are VeryQuickWiki. Have you ever talked with those folks. You should merge the projects someday, IMHO -- PhilippeO

-- I looked at VeryQuickWiki when I was looking. The features in JSPWiki are ahead of VQW. There are some nice things in VQW like the admin.jsp and how you can add Virtual Wiki to the base. The providers in JSPWiki are better, as is plugin support. I'm looking forward to seeing your XMLFile Provider, I have another project that is interested in using XML as a searchable data structure. -- FosterSchucker -- Good feedback on vqwiki, they have also made some interesting enhancements like using Lucene for search, which is industrial strngth serach & indexing stuff. I actually plan to use Lucene with the IsoGen XML indexing extensions, to be able to query the set of XML wiki pages. This is not Xquery, but there is not yet a complete OSS Xquery implementaion out there, to my knowledge. But the combo does the trick nicely, and should provide decent structured search on strcuture XML wiki pages.

On another note, about the virtual wiki, I like very much another approach to that developped in FitNesse, where any page can become a virtual/sub wiki of sorts, with subpages. You therefore implicitely reproduce a tree, which -at least conceptually- become very similar to an XML structure, albeit a level above, more similar to a hierarchy of folders in a file system. I like it. There has been some FitNesse discussion on this wiki already back in February 2003. -- PhilippeO

Take a look at the attached SimpleNoteConnection. Its the code we use to store our notes in XML.
- YishayMor

-- Thanks. I will check your code out. For now, I am looking at using Dom4J . Just curious : what is weblabs.org? -- PhilippeO

Well, its actualy http://www.weblabs.eu.com/. Should be weblabs.edu or something, 'cause its a research project. Ping me if you want to hear more.
- YishayMor

I'd like to see not XMLStorage as such - but XML output - for instance getting the whole of the Wiki out as a big XML file which I could then turn into (for example) a PDF or OpenOffice document. -- AlexMcLintock Jul 2003
docbook style

Don't make it the default though, the current format of the basic text pages is perfect. One people I've had with all wiki's so far is their incompatable storage systems.

-- BrillPappin July 2004

I have a site http://www.encyclopaedia-wot.org that has a few thousand pages in it, all hand created. I want to turn them into XML and use XSL to display them. I have my own XML schemas. I also want my staff of volunteers to be able to edit the pages when necessary, just like Wikis. And, I want to have the Wiki-style links embedded in the XML, because my staff shouldn't have to memorize all the URLs that each page links to. It sounds like what you're trying to do here is what I need. Plase keep up the work.

Gary Kephart - Feb 2005

I've implemented an Xindice-based XML database within my own application, and I have never seen any problem in storing non-XML content within it. You merely wrap the content in a CDATA section and many of the above issues go away. You can then put that CDATA section into a SOAP-like wrapper, where you can add your document metadata. In that wrapper could also be every other revision of that wiki page. The wrapper could hold plaintext, wiki text, HTML, XHTML, or any flavour of XML. (Binary content could also be encoded and included, but that doesn't seem like a very efficient approach.)

I wrote up just such a wrapper for my application and called it XNode, and have been using it for the past three years. I've been thinking quite seriously about implementing this as an alternative JSPWiki backend, as it would work fine. The biggest issue for me is time -- the next few months for me look to be quite hectic, and I'd have to pull the Xindice backend out of my own application to make the code available to JSPWiki.

If anyone is interested in pursuing this, I'd be happy to post the XNode API online, and perhaps also contribute code.

MurrayAltheim - August 2005

Category Ideas

Help and formal document formats are consumed by endusers and management. Wikis by their very nature are loosely structured and do not meet this need. It would make sense to set up JSPWiki content for DocBook output so you can meet the need of these audiences. I would like to generate my output directly from the database. First, I would like to create developer documentation using one query and then another for end user consumption. Developer content tends to be granular whereas end user versions require guidance and hand holding. Finally, build a path to DocBook to accomplish these ends and the world will beat a path to JSPWiki. This will probably happen now that Sun has released Creator 2 and has enabled Java distro bundling with its new license strategy.

--Sumner, 17-May-2006

Add new attachment

Only authorized users are allowed to upload new attachments.

List of attachments

Kind Attachment Name Size Version Date Modified Author Change note
MyWikiView.xml 2.6 kB 1 11-May-2004 06:09
SimpleNoteConnection.java 3.1 kB 1 06-May-2003 12:49 YishayMor
WikiView.xsl 18.6 kB 1 11-May-2004 06:09
« This page (revision-30) was last changed on 07-Jan-2007 09:47 by MurrayAltheim