PreviousNextTracker indexSee it online !

(14/211) 2822 - Editing files with 100000+ character lines is impossible

Hi,

When you edit a MySQL dump (~2600 lines / 13.8 Mo), jEdit freeze with 100% CPU.
I think this is due to very long lines (1 insert statement per table).

This is very annoying and I must use an other text editor.

--Marc

Submitted mdenty - 2007-02-21 13:52:38 Assigned kpouer
Priority 5 Labels editor core
Status open Group normal bug
Resolution None

Comments

2007-02-21 14:09:43
mdenty

Logged In: YES
user_id=84736
Originator: YES

This test was done under jEdit 4.3pre9

2007-02-21 14:09:43
mdenty

- **priority**: 5 --> 7

2007-05-14 15:33:24
kpouer

Logged In: YES
user_id=285591
Originator: NO

Do you have an OutOfMemoryError in the activity log ?

2007-05-14 15:58:15
mdenty

Logged In: YES
user_id=84736
Originator: YES

It's difficult to say since Jedit is frozen after the bug triggers, but JEdit eats 100% of one CPU.
Here is the stack trace given by jconsole :
Name: AWT-EventQueue-0
State: RUNNABLE
Total blocked: 133 686 Total waited: 766 480

Stack trace:
org.gjt.sp.jedit.TextUtilities.getTokenAtOffset(TextUtilities.java:73)
org.gjt.sp.jedit.TextUtilities.findMatchingBracket(TextUtilities.java:207)
org.gjt.sp.jedit.textarea.StructureMatcher$BracketMatcher.getMatch(StructureMatcher.java:70)
org.gjt.sp.jedit.textarea.TextArea.updateStructureHighlight(TextArea.java:5510)
org.gjt.sp.jedit.textarea.TextArea.access$300(TextArea.java:65)
org.gjt.sp.jedit.textarea.TextArea$1.actionPerformed(TextArea.java:6042)
javax.swing.Timer.fireActionPerformed(Timer.java:271)
javax.swing.Timer$DoPostEvent.run(Timer.java:201)
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:183)
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:173)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)
java.awt.EventDispatchThread.run(EventDispatchThread.java:121)

Hope this help,

--Marc

2007-06-15 05:06:46
vanza

Logged In: YES
user_id=75113
Originator: NO

This doesn't really need to be that severe. It's a limitation of the current code.

I started a new branch to try to minimize copying of strings, and editing files with long lines is MUCH MUCH faster with the current code in that branch. The code doesn't work 100% correctly yet, though (so don't try it at home\!), and I haven't had much time to play with it.

2007-06-15 05:06:46
vanza

- **milestone**: 101607 --> normal bug
- **priority**: 7 --> 5

2007-07-30 19:25:59
kpouer

Logged In: YES
user_id=285591
Originator: NO

Hi I opened bigger files, could you try to open such file with the text edit mode ? (maybe rename the file so it doesn't match any glob in the catalog)

2007-07-31 07:22:49
mdenty

Logged In: YES
user_id=84736
Originator: YES

Hi,

This does not change anything since it seems to be caused by very long lines (1000000 chars).

--Marc

2007-07-31 19:10:57
ezust

- **summary**: Editing MySQL dumps is impossible. --> Editing files with 1000000+ character lines is impossible

2007-07-31 19:10:57
ezust

Logged In: YES
user_id=935841
Originator: NO

Are you using "soft", "hard" or no line wrap? I bet that for soft wrap, it would perform quite badly, but perhaps if you are not wrapping lines, it won't need to do as much work... not that you'd want to use the horizontal scrollbar to see a 1000000 character line, but this could narrow down the problem. I'm changing the subject line to be more descriptive.

2007-07-31 19:11:25
ezust

- **summary**: Editing files with 1000000+ character lines is impossible --> Editing files with 100000+ character lines is impossible

2007-08-01 07:02:24
mdenty

Logged In: YES
user_id=84736
Originator: YES

Hi,
By default line wrapping is at "none" state. (freezing JEdit when moving the cursor)
I tried "hard" state, JEdit froze immediately.
I tried "soft" state, JEdit reformat the sql queries but when I move the cursor for few lines it froze too.

Usually I use "none" since on a such file I only do search/regexp search and I don't need to browse it all.

--Marc

2011-12-04 17:48:55
ezust

- **assigned_to**: nobody --> kpouer

2011-12-04 21:17:07
kpouer

I have a solution that is not very good :
the problem is that syntax highlight is slow on so many text, the solution is that when we find a very long line we can use the text edit mode for that line.
This makes jEdit fast, the problem is that it will break the syntax highlight for that line, and probably for the next lines

2011-12-05 07:41:27
jarekczek

Or the similar way:
On/after loading of a file with at least one long line give a warning and suggest to switch to text mode. One of the buttons would do it automatically.

I don't think anyone really needs syntax highlighting in these machine generated files.

2011-12-05 09:21:45
kpouer

In fact what I did is to switch to context insensitive highlight so at least the previous lines are correctly highlighted, and the next lines may be highlighted as they should in several languages

2017-12-03 00:08:10.616000
tsourick

All the options like switching to text mode or relaxed highlighting are now implemented (see "large files" section of global options) and the dialog pops up on file open.

Besides, I have no problem editing 20Mb+ MySQL dump in 2017 with all those corei7, ddr4 etc.

A good reason for closing...