In version 0.26 there was a major change in interpretation of newlines. In previous version a newline was almost always translated into <br />, but in the new version paragraphs has been correctly tagged with surronding <p>-tags. This, however, raised another issue as to what does a single newline at the end of a paragrap mean.

To illustrate the issue consider the following example, with some different interpretations on how to display this:

Line 1, paragraph 1.
Line 2, paragraph 2.

Option 1: In version 0.25 this is displayed/coded as two separate paragraphs:

<p>Line 1, paragraph 1. </p>

<p>Line 2, paragraph 2. </p>

Option 2: Another option is to interpret the newline as a line breaker, like the newly introduced \\-operator:

<p>Line 1, paragraph 1.<br />
Line 2, paragraph 2. </p>

Option 3: A third option, is to allow for faulty browsers which include newlines from textboxes, to still have proper paragraph coding. This would lead to the interpretation of one newline is actually a line continuation if followed by a 'paragraph' line, and to separate paragraphs one need two following newlines. Anyway, then the example would be coded as:

<p>Line 1, paragraph 1. Line 2, paragraph 2. </p>
Are there any browsers still in use today that show this faulty behaviour (introduce "real" newlines for "virtual" line breaks on wrapped long lines)?

Run discussion

As can be seen, there are several options on how to interpret this. Please add your comments onto this page, and the developers are open to changing this into what the community wants and understands as the best option.

EvenHolen: I'm thinking that I kind of like the way it's just now. We have the options to join lines using \, and to introduce line breaks using \\. So my thought is that if we want to change this, it should be to automagically join lines into the same paragraph, option #3. But then again this could lead to false paragraph.

RichardCyganiak: Make it simple for authors. When I insert a line break, I expect the rendered page to have a line break in the same place. When Tavi inserts a full line of whitespace instead, or just ignores the line break, I consider this to be broken. Try to think like a naive author who is new to Tavi. To him, option 2 is the only one that makes sense.

I respectfully disagree that #2 is the "only one that makes sense". If you're typing a plain-text email and you write two lines with no space between, you usually intend them to be part of the same paragraph.

Probably not if you wrap in column 10. Maybe if you're near column 80, because your email app doesn't do smart word wrapping with quoted text, but that's a non-issue with Tavi.

In fact, most wikis do not behave like option #2. In most wikis, two adjacent lines (no blank lines separating them) are strung together in a single paragraph (like option #3). Try it on WikiWiki or in UseModWiki. WikkiTikkiTavi's original behavior (#2) is different from this, and if I had it to do over again I would have made it like WikiWiki and UseMod. Actually, I did do it over again for [Jot], and I'm much pleased with the results. -- ScottMoonen

But why is this better than #2?

TimHogan: Well since we are voicing our opinions, here is mine. I have worked with text based environments for years now, mostly unix of some sort, and I've typed a lot of notes in vi. When I type up a set of notes or instructions in vi I expect the line breaks to be where I put them for readability. If I type a line and hit one return then type another line of text I do not expect to see any blank line in between. If I wanted a blank line I would have hit enter twice. Now I understand that a wiki is not a word processor or a text editor per say, but my opinion would be to make the wiki behave along those lines when it comes to line brakes. Prime example, click on my name, there should not be white space in between those lines but there is. Regards...

RichardCyganiak: #1 and #3 make it impossible to type up a postal address without using syntax hacks. #2 is closest to WYSIWYG, and adheres to the priniciple of least surprise. #2 is how most Word users format their documents (yes I know that's not how Word is supposed to be used). If you advocate #3 or especially #1, please try to see the issue from the point of view of someone who has not used a wiki before and has not much computer experience in general. Tavi is great for those people because of its clean design (better than any other wiki engine I know), but the change from #2 to #1 was a step backwards.

I disagree with the MS Word argument, since they have made a lot of illogical coding, and it's not a clean good logical solution to text typing. But otherwise you make a good point.

And on a side note: I see that RC's is a co-maintainer on a ZWiki-implementation, which also adheres to #3 ;)
--EvenHolen
I disagree with the ZWiki argument, since they have made a lot of illogical coding ;-)

EvenHolen: A new approach could be to implement both #2 and #3?! I see RC's and TH's point about WYSIWYG, but I still mostly agree with Scott. So one new approach could be to add <br /> if length of line is below a threshold, say 30 characters, and otherwise join the lines? (And let the threshold be controlled in config.php. And then again, maybe this isn't a good approach either as it would be ambigious, since similar text would produce different markup. Hmm... it's hard to decide on what the proper approach would be...

I dunno. Sounds like it could cause some confusion.
I like this solution a lot. The threshold setting would let people decide how they want there wiki to work. I can see the reasons for going with option 3 but for my own wiki option 2 is better. -- FredrikJonsson

The case against #3

RichardCyganiak: I'm trying to understand why #3 is so attractive to Even and Scott. Here are the advantages I can see:

Anything else?

The main disadvantage of #3 is that some very simple things become very hard:

In these cases, the result will be a total mess. #3 doesn't "fail gracefully", it fails by messing up your text, and you can fix it only if you know about the backslash syntax. Contrast to #2, which fails by displaying some line breaks that you have put there but didn't actually want, and it's obvious how you can fix your text.

To summarize: #3 makes it hard for clueless authors to produce messed up paragraphs with lots of unnecessary line breaks and saves some time when copy&pasting text, but also makes it impossible for clueless authors to do some very basic things. #3 is appropriate and powerful in environments with authors who are familiar with the finer points of the WikkiTikkiTavi syntax, but will cause trouble in an environment with authors who just want to publish/edit some text with a minimum of learning.

I care a lot about environments of the second kind. That's why I feel so strongly that #3 is not a good solution.

EvenHolen: Regarding disadvantages, I see your point regarding postal adresses and poems. And I ask: Which percent of pages/paragraphs are this kind of information? You don't need to answer, but there is a followup question: Given this (I suspect) rather low percentage, is it worth it to break having proper paragraphs in the rest of the documents?
RichardCyganiak: Postal addresses might be rare on your wiki, and common on mine. And how does #2 break having proper paragraphs? I don't advocate the old "translate every newline to <br>" solution. You can have proper paragraphs and logical HTML markup with either of the suggested solutions (for #2, transform single newlines to <br> and double newlines to <p>).
I'm a firm believer of HTML being logical markup, and the more I think about it, the more it makes sense to have paragraphs be formed automatically. Regarding the code inserts, I believe this to be an issue of education and making it simpler to get the help on formatting rules. (And I will adress this by optionally providing a small box of formatting rules beneath the edit field. This I think to be turned on/off on a user basis, defaulting to on for new users)
A box with some formatting rules below the edit area would be a VeryGoodThing?, but kind of orthogonal to the issues discussed here. But since you bring it up: Please note that, when using #2, there would be no need to mention newlines in this box, while with #1 and #3 you'd have to explain the backslash syntax. In my world, this alone would be almost enough to disqualify #1 and #3.

MikeWagner: I've implemented tavi for a couple groups of people and they love it. however, they were not happy when i upgraded to the latest version and all of their extensive number of pages were formatted differently than they were the day before. all the line formatting on hundreds of pages was changed. the only way to fix it is to add \\ to hundreds of lines... or, is there another, easier way?

EvenHolen: In version 0.25 there a no way to re-introduce the old paragraph behaviour. But pending on this discussion, maybe things will look better in the next. At the time being, I'm still leaning towards #3, and that does, unfortunately, involve some changes on some old sites. But I still believe that mainly newlines has been used within paragraphs, and not in the special cases of poems, adresses or code inserts. As an example, look at most pages here at this site. There were a few which need refactoring, but not more than a handful. Please show me examples to the contrary.
PaH: Contrary, well, It happened on my site, as the old formatting was not applied, and I have today Refactored some pages on this Wiki, I hope I haven't messed up everything to other people. The reason for why I refactored them is that I had problem reading them. On top of this page it says "in version 0.26 there was..." will it be better to say: it will be, or is it a typo meaning 0.25? Here is my page http://www.vikinc.net/wiki/ I wonder what I will do, adding those \ \ or maybe experimenting with the source-code to change the behavior on my site. I guess I just hacked the source-code, parse/html I also have to reimplementing the URL-Rewrite-code, since after upgrading from my customised version the Tavi did not work on my ISP-webspace. but that has no pri 1 :)
Ahh those \ \ :), i have to remember them ...
I like it when what you see in Edit mode is close to 100% what you get in View-mode.
Therefore my website is now changed to reimplement the br-codes.
copy paste from text-files is great, and implementing, <code> or <phpcode> might remove functions of '''. *. # and such nice elements And finally here are just a few examples... where I found <p> hacking necessary
(2004.04.12 05:16:41) (history) RickVanLieshout/InitDatabasePHPScript . .(Petter Arild)? = PAH
(2004.04.12 04:54:43) (history) DietrichKracht . . . . (Petter Arild)?
(2004.04.12 03:24:31) (history) NoExecuteOnServer . . . . (Petter Arild)?
(2004.04.12 03:05:07) (history) TaviOnWindows . . . . (Petter Arild)?
Best regards ... PAH

Why not selecting the behavoir with a Variable in config.php?

As long as #2 is the default ;-) RC

And how could I make #1 the default?

Qualidafial: I vote strongly for #2. The new interpretation of newlines has made 'Tavi more difficult to use. After using 0.23 and 0.24 for some time, I upgraded to 0.25 and suddenly all my pages were screwed up. On most of my pages, groups of paragraphs that logically belonged together were now spaced apart, and the visual grouping disappeared. Eventually I got so frustrated that I hacked the code and changed it back to the old formatting.
There are a million ways to use a wiki, and not all of them require nice, neat paragraphs. Not everybody out there is going for the same look. Poetry and addresses are not as uncommon as you think (some poetry has even found its way onto our corporate wiki). Doesn't it make more sense to have the html page look just like the edit page? When I'm authoring pages, I want my pages to look a certain way with a minimum of fuss.
The great strength of 'Tavi is/was the intuitive interface. I was able to just pick it up and use it. The new formatting is not intuitive. Nobody looking at the page gives a crap about the HTML. But they WILL if they get lost in a page where logical grouping of ideas is lost. Even this page, if you edit it, has blank lines between paragraphs. Why? Because it's easier to read that way.
This whole debate is happening because somebody decided that <p> tags are better. Why? Odds are that people are going to separate their paragraphs with empty lines anyway. So why force the rest of us to jump through hoops just to get a fricken line break? Let's make it easy for authors to achieve the look they want.
P.S. Please include my patch for text-enhance in the next release. The patterns can get confused if you put a single slash anywhere between the pair of double-slashes. The same applies to all other forms of formatting markup. --see TaviPatches/TextEnhance.

[By-Tor]: Wow, I upgraded to 0.25, and is busted all my pages in Tavi. And yes, I do say Busted. Please make a configuration option to revert to the old 0.24 behaviour in regards to rendering pages. Pages should appear as they are inputted. The change to <P> tags is an unacceptable rendering of the pages. What I see is not what I put into the wikki. I'm downgrading to 0.24 untill this issue is resolved.

[By-Tor]: Well, I had to upgrade to 0.26 for mysql 4 support. So here is how I solved it for me:

Change function parse_newline($text) tavi-0.26/parse/transforms.php

from:

 return preg_replace("/^(([^$FlgChr]|$FlgChr!).+)$/e",
                     "pair_tokens('paragraph', q1('\\1'), true)", $text, -1);

to:

   return preg_replace("/\\n(\\r)?/e", "new_entity(array('newline'))",
                         $text, -1);

Basically, this preg_replace was copied directly from the same function from tavi-0.24. Hopefully I didn't break anything ;)

[vlko]: I solved this problem this way:

in function parse_newline($text) at transforms.php replace

  return preg_replace("/^(([^$FlgChr]|$FlgChr!).+)$/e",
                      "pair_tokens('paragraph', q1('\\1'), true)", $text, -1);

with

  return preg_replace(
    array("/^(([^\\n]!|[^$FlgChr]|$FlgChr!).*)[\\n]+$/e","/\\n(\\r)?/e"),
    array("pair_tokens('paragraph', q1('\\1'), true)","new_entity(array('newline'))"),
                      $text, -1); 

now everithing looks fine:)

and of course don't forget to place to your .css style file, if you want to disable adding margins

p
{
  margin: 0.1em;
} 
Yes, please, make an option to select behaviour in configuration. I love tavi beacuse of its simplicity, but in constant use over number of projects i've seen it's so much logical to translate all the newlines into <br>. especialy if you use tavi more like a scrapbook for example and sometimes want to have a lot of white space when structuring text. -- LukaPrincic