# Visual basic text file nightmare



## shuggans (Mar 28, 2011)

Hello, I am writing a program that needs to be able to read from a text file and display its output into a multiline text  box.  I have this part done.   my problem is, I need this to update in real time as the text file im reading from is a log that keeps track of uploading of files etc.  the method i had had in mind to update the text box in real time was to loop the reading of the text file and add a thread.sleep(5000) line in the loop, giving 5000 milliseconds before it refreshes- this method causes the program to freeze.  any other ideas on how to do this?  Thanks in advance
-sean


----------



## temp02 (Mar 28, 2011)

It has been  while since I've messed with Visual Basic but I'm pretty sure there is a visual component called Timer (or something equivalent), add one to the form, double click on it and add your "loop" code, now without the loop, to it, then set the timer interval property to 5000 and your set.


----------



## W1zzard (Mar 28, 2011)

read up on FindFirstChangeNotification and related functions, for the correct way to do this (in a second thread)

what is probably easier for you is using settimer to periodically get a timer called


----------



## FordGT90Concept (Mar 28, 2011)

If it were me, I'd make a separate class with the reader in it then have it raise an event whenever new lines were detected (each event raised would contain the line).  That way, the GUI only updates when it needs to.


----------



## Kreij (Mar 28, 2011)

You didn't mention what application is doing the writing to the text file, but you will probably want to implement error checking in there when reading the file. 
If the app doing the writing has an exclusive lock on the file that prevents any other app from accessing it for either reading or writing, your app will throw access denied exceptions when trying to open it for a read.

_Disclaimer : It's Monday_


----------



## Akumos (Mar 29, 2011)

Kreij said:


> You didn't mention what application is doing the writing to the text file, but you will probably want to implement error checking in there when reading the file.
> If the app doing the writing has an exclusive lock on the file that prevents any other app from accessing it for either reading or writing, your app will throw access denied exceptions when trying to open it for a read.
> 
> _Disclaimer : It's Monday_



Been a while since I monitored this topic, I don't think I ever got around to saying... THANK YOU KREIJ for helping me graduate university!!!


----------



## FordGT90Concept (Mar 29, 2011)

FYI, you can work around file locks by having CMD do a copy.  CMD copy ignores file locks.  Read the copied file, then delete the copy when done.


----------



## CrackerJack (Apr 1, 2011)

Hope i'm not to late... But like said by others

Use a timer. set the internal to 5000.... if it still cause freezes. I would try "backgroundworker" to grab the text file, and then have timer to read it...


----------



## smartali89 (Apr 1, 2011)

Application.DoEvents()

This might solve the problem, Which version of VB are you using?


----------



## Kreij (Apr 2, 2011)

Akumos said:


> Been a while since I monitored this topic, I don't think I ever got around to saying... THANK YOU KREIJ for helping me graduate university!!!



My pleasure, Akumos, always happy to help any way I am able. 
Congrats on your graduation (I can't believe you didn't invite me to your graduation party. :shadedshu ) and I think I will add "Helped people graduate" to my resume. lol


----------



## shuggans (Aug 14, 2011)

Sorry for the long response, but I am just now getting back to this abandoned part in the project -
I have resorted to just creating this in its own seperate app till I get this working before implementing it in the project.  SO... I have some code for you guys.  This is the best thing I have got so far... IF you use this in a consoel application and change the line that outputs this to the textbox to "console.writeline(line)" it works flawlessly.  But as a windows forms application... nooot so much. try it and see.
-----------------------------------------

Imports System.IO

Public Class Form1

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Using reader As New StreamReader(New FileStream("C:\logtest\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            'start at the end of the file
            Dim lastMaxOffset As Long = reader.BaseStream.Length

            Do
                System.Threading.Thread.Sleep(100)

                'if the file size has not changed, idle
                If reader.BaseStream.Length = lastMaxOffset Then
                    Continue Do
                End If

                'seek to the last max offset
                reader.BaseStream.Seek(lastMaxOffset, SeekOrigin.Begin)

                'read out of the file until the EOF
                Dim line As String = ""
                line = reader.ReadLine()
                Do While line IsNot Nothing
                    TextBox1.AppendText(line)
                    line = reader.ReadLine()

                Loop

                'update the last max offset
                lastMaxOffset = reader.BaseStream.Position
            Loop

        End Using
    End Sub
End Class

Edit: P.s. Using Visual Studio 2010 Ultimate


----------



## CrackerJack (Aug 14, 2011)

If your only reading the first line of the text file use something like this... I don't see the point of running a loop if only the first line is been read, But of coarse this will read the line no matter what.


```
Using reader As New StreamReader("C:\logtest\test.txt")
            'Reading only the first line in test.txt
            Dim line As String = ""
            line = reader.ReadLine()
            TextBox1.AppendText(line)
        End Using
```

now to have to it read it every 5000ms (5sec)


```
Imports System.IO

Public Class Main

    Private Sub Main_Load(sender As System.Object, e As System.EventArgs) Handles MyBase.Load
        ReaderTimer.Start()
    End Sub
    Private Sub ReaderTimer_Tick(sender As System.Object, e As System.EventArgs) Handles ReaderTimer.Tick
        ReaderTimer.Interval = 5000
        Using reader As New StreamReader("C:\logtest\test.txt")
            'Reading only the first line in test.txt
            Dim line As String = ""
            line = reader.ReadLine()
            TextBox1.AppendText(line)
        End Using
    End Sub
End Class
```


----------



## Kreij (Aug 14, 2011)

He's not reading the first line. He's reading any new lines that have been appended to the end of the file since the last read.

Something you may want to consider, shuggans, is append the new lines to the top of the TextBox so that the newest entry in the file is always at the top.
You could then truncate the contents of the textbox if it gets longer than you want.

Just a suggestion.


----------



## CrackerJack (Aug 14, 2011)

Kreij said:


> He's not reading the first line. He's reading any new lines that have been appended to the end of the file since the last read.
> 
> Something you may want to consider, shuggans, is append the new lines to the top of the TextBox so that the newest entry in the file is always at the top.
> You could then truncate the contents of the textbox if it gets longer than you want.
> ...



ah yea i didn't catch that


----------



## shuggans (Aug 14, 2011)

I need to have the whole log available to view-
The log isnt updated all that often, and the point of it in the program is so you never have to hunt the log down and open it in notepad.  

would it be better to use something other than a textbox to do this, such as a combobox?  

also, crackerjack, I haven't  used timers since my IRC script kiddie days.  Certainly never in VB before, how do I properly define the timer used in the code you posted, and I could try just modifying it to readtoend


----------



## FordGT90Concept (Aug 14, 2011)

You never want to read-to-end because it is inefficient in every regard (unless it is a very, very small file).  What you'll want to do is keep record of the offset of your last read and whenever the file length changes, jump to the last offset then read the lines until the end of file (rince and repeat).

I would probably use a ListBox--one item per line--perhaps bound to a stack to always place the most recent item at the top.


----------



## shuggans (Aug 14, 2011)

and kreij, If you try the code I had as a console app, it works great.  In a windows form, it causes the app to become unresponsive.  and idea why?  this has to be possible to do...
Is there a way to embed a console into a vb.net form?


----------



## FordGT90Concept (Aug 14, 2011)

If the application ever becomes unresponsive, it is because something is tying up the main thread that needs to be moved to a worker thread.

No, you can simulate a console in a Windows Form but they are different (CUI vs GUI) on an executable level.


How many files are you trying to monitor?  Just one?


----------



## shuggans (Aug 14, 2011)

there will be two separate txt files monitored in two separate text or combo boxes side by side.


----------



## Kreij (Aug 14, 2011)

Since your loop is in the main UI thread it's causing the UI to be unresponsive.
Put that in a worker thread as Ford stated.

I see some issues with maintaining an offset on repeated runs of the app, as if the logfile gets deleted or truncated, the offset will be wrong.


----------



## shuggans (Aug 14, 2011)

This is my firs whack at multi threading, what did I do wrong, form still locks up:

Imports System.IO
Imports System.Threading

Public Class Form1


    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        readlog()
    End Sub

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

    End Sub
    Public Sub readlog()
        Dim Thread_UploadLogReader As New System.Threading.Thread(AddressOf readlog)
        Thread_UploadLogReader.Start()
        Using reader As New StreamReader(New FileStream("C:\logtest\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            'start at the end of the file
            Dim lastMaxOffset As Long = reader.BaseStream.Length

            Do
                System.Threading.Thread.Sleep(100)

                'if the file size has not changed, idle
                If reader.BaseStream.Length = lastMaxOffset Then
                    Continue Do
                End If

                'seek to the last max offset
                reader.BaseStream.Seek(lastMaxOffset, SeekOrigin.Begin)

                'read out of the file until the EOF
                Dim line As String = ""
                line = reader.ReadLine()
                Do While line IsNot Nothing
                    TextBox1.AppendText(line)
                    line = reader.ReadLine(line)
                Loop

                'update the last max offset
                lastMaxOffset = reader.BaseStream.Position
            Loop

        End Using
    End Sub
End Class


----------



## CrackerJack (Aug 14, 2011)

Please wrap your code 

I believe i see one issue, updating VS atm... so gimme a few till i can post back


----------



## FordGT90Concept (Aug 14, 2011)

Kreij said:


> I see some issues with maintaining an offset on repeated runs of the app, as if the logfile gets deleted or truncated, the offset will be wrong.


If file.Length < lastoffset Then
  ' Start over
End If

Edit: I'm trying to think if I already coded a generic line parser but I don't think I did.  They were always specific to a format (broke down every line into parts) so nothing I have would work out of the box.  It really wouldn't be hard to make one though.


----------



## Kreij (Aug 14, 2011)

FordGT90Concept said:


> If file.Length < lastoffset Then
> ' Start over
> End If



What if the logfile was deleted and has grown longer than the offset already?


----------



## FordGT90Concept (Aug 14, 2011)

That would result in an "oops."  That issue is negated by more frequently checking the size (like 100ms instead of 5000ms).

You can also go by date-modified too but, in this situation, I'd perfer length so if the application that's writing the log just touches the file (opens a write stream and closes it), it doesn't invoke an update.


----------



## Kreij (Aug 14, 2011)

@shuggans, 
Sorry for going off-topic. lol
It looks like you are starting a thread but still running the loop in your main method.


----------



## CrackerJack (Aug 14, 2011)

Still can't figure it out, But you have the thread start wrong....


```
Imports System.IO
Imports System.Threading

Public Class Form1


Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
'Thread start should be here, other wise it's repeat it self over and over till stop
readlog() 
'Dim Thread_UploadLogReader As New System.Threading.Thread(AddressOf readlog)
'Thread_UploadLogReader.Start()
End Sub

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

End Sub
Public Sub readlog()
Using reader As New StreamReader(New FileStream("C:\logtest\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
'start at the end of the file
Dim lastMaxOffset As Long = reader.BaseStream.Length

Do
System.Threading.Thread.Sleep(100)

'if the file size has not changed, idle
If reader.BaseStream.Length = lastMaxOffset Then
Continue Do
End If

'seek to the last max offset
reader.BaseStream.Seek(lastMaxOffset, SeekOrigin.Begin)

'read out of the file until the EOF
Dim line As String = ""
line = reader.ReadLine()
Do While line IsNot Nothing
TextBox1.AppendText(line)
line = reader.ReadLine(line)
Loop

'update the last max offset
lastMaxOffset = reader.BaseStream.Position
Loop

End Using
End Sub
End Class
```


----------



## shuggans (Aug 15, 2011)

*Gahhh*

Ive messed with this for so long.  Is this even possible to do in a windows form?  Moving the thread start to the button click event produced the same results.


----------



## FordGT90Concept (Aug 15, 2011)

I wrote a class quickly to do what you are asking for with an example form of how to implement it...


Basically, add MonitorFileForChanges.vb to your project then, for each file you want to monitor, do something like:


```
Private WithEvents file1 As New MonitorFileForChanges("Path to file1")
Private WithEvents file2 As New MonitorFileForChanges("Path to file2")
Private WithEvents file3 As New MonitorFileForChanges("Path to file3")
```

Next, you'll have to start the workers.  This is done by simply calling file1.StartWorker(), file2.StartWorker(), and file3.StartWorker().

The toughest part is avoiding that nasty cross-thread reference. The code below basically detects if a thread that isn't itself is accessing the method and if so, it runs it from itself (yeah, that's complex). I bolded the bits that need to change for each one.  You should use more descriptive names than file1, file2, and file3, obviously:


```
Public Sub [b]File1[/b]_LineAdded(ByVal line) Handles [b]file1[/b].LineAdded
        If Me.InvokeRequired Then
            Me.Invoke(New LineAddedHandler(AddressOf [b]File1[/b]_LineAdded), line)
        Else
            [b]ListBox1.Items[/b].Add(line)
        End If
    End Sub

    Public Sub [b]File2[/b]_LineAdded(ByVal line) Handles [b]file2[/b].LineAdded
        If Me.InvokeRequired Then
            Me.Invoke(New LineAddedHandler(AddressOf [b]File2[/b]_LineAdded), line)
        Else
            [b]ListBox2.Items[/b].Add(line)
        End If
    End Sub

    Public Sub [b]File3[/b]_LineAdded(ByVal line) Handles [b]file3[/b].LineAdded
        If Me.InvokeRequired Then
            Me.Invoke(New LineAddedHandler(AddressOf [b]File3[/b]_LineAdded), line)
        Else
            [b]ListBox3.Items[/b].Add(line)
        End If
    End Sub
```

Finally, those worker threads work in a more-or-less infinite loop.  This means you have to stop them or else your application won't close when it is told to close.  You can do this by creating a handle to the FormClosing event and then call file1.StopWorker(), file2.StopWorker(), and file3.StopWorker() like so:


```
Private Sub Form1_FormClosing(ByVal sender As System.Object, ByVal e As System.Windows.Forms.FormClosingEventArgs) Handles MyBase.FormClosing
        file1.StopWorker()
        file2.StopWorker()
        file3.StopWorker()
    End Sub
```

Now, if a line is added to any of three files, it will show up in ListBox1-3.


----------



## Kreij (Aug 15, 2011)

Nice Ford. I thought you quit coding for awhile to play games.


----------



## FordGT90Concept (Aug 15, 2011)

Beyond Good & Evil lost its pull on me.  

3 more days to From Dust...


----------



## shuggans (Aug 15, 2011)

temp02 said:


> It has been  while since I've messed with Visual Basic but I'm pretty sure there is a visual component called Timer (or something equivalent), add one to the form, double click on it and add your "loop" code, now without the loop, to it, then set the timer interval property to 5000 and your set.



Wow.  Just tried abandoning my code and trying this...


```
Imports System.IO
Imports System.Threading

Public Class Form1


    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

    End Sub


    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        Timer1.Interval = 100
        Timer1.Start()
    End Sub
    Private Sub Timer1_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Timer1.Tick
        Using reader As New StreamReader(New FileStream("C:\logs\UploadLog.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            TextBox1.Text = reader.ReadToEnd
        End Using

    End Sub


End Class
```

And it works beautifully.  Sorry to take up your time Ford :s.
Two questions with this though
1. Am I correct in thinking each timer uses and starts its own thread automatically?
2. How do I lock the scrollbars on a textbox to the bottom of the textbox?


----------



## shuggans (Aug 15, 2011)

*Question about multithreading in vb.net*

When I start A new thread for a subroutine to run on, should I start the new thread in that subroutine, or in the event that calls the subroutine?


----------



## FordGT90Concept (Aug 15, 2011)

1. Yes.
2. After updating the text...

```
TextBox1.SelectionStart = TextBox1.Text.Length
TextBox1.ScrollToCaret()
```

Just beware that ReadToEnd() and using a TextBox will start to slow down after about 32 KiB of data are read.


----------



## FordGT90Concept (Aug 15, 2011)

The form is always on the "main" thread.  It will own the thread(s) it starts.

Usually you don't start threads inside of threads unless they are doing something special/different.

You need to start work threads from somewhere in the main thread.  It doesn't particularlly matter where.

You'll get circular logic if you try to start a thread from inside itself.  In fact, it would probably never even start in the first place.


----------



## shuggans (Aug 21, 2011)

*Write to a text file without streamwriter?*

Ok.  This is the last piece holding me up.  
Is there ANY way I can write to a text file without locking it in use?  Im currently using streamwriter- I need a different method to do so.  Is this possible?  would using a cmd to append a string be the best way, and if so is  there a way to do it without causing a CMD window to pop up??  and what are some more alternatives?  
 Thanks in advance


----------



## FordGT90Concept (Aug 22, 2011)

When you make a new FileStream, one of the later options in it is the file locks.
http://msdn.microsoft.com/en-us/library/ms143397.aspx

FileShare is the one you need to modify:
http://msdn.microsoft.com/en-us/library/system.io.fileshare.aspx

Example (substitute "StreamWriter" for "StreamReader"):
http://balajiramesh.wordpress.com/2008/07/16/using-streamreader-without-locking-the-file-in-c/


If that doesn't work then yeah, you'll have to look into cmd options.


----------

