Replacing text in a stream after writing

Recently I responded to a question asked on StackOverflow. The question was "What is the BEST way to replace text in a File using C# / .NET?"

Just by the question you might assume they were wanting to simply replace text in an existing file and of course you could do that by opening the file and looping over it or doing a Replace of content.

Here is the requirements they gave.

Have a text file that is being written to as part of a very large data extract. The first line of the text file is the number of "accounts" extracted.

Because of the nature of this extract, that number is not known until the very end of the process, but the file can be large (a few hundred megs).

What is the BEST way in C# / .NET to open a file (in this case a simple text file), and replace the data that is in the first "line" of text?

So as you can see they are in the process of writing the file, at the end of the write of the file they now know the information they wish to write at the top of the file. So how might we able to go back and write that information without having to open the file again and write the information at the top of the file?

It turns out it is actually fairly simple. One we want to make sure memory utilization is a factor since these files can be very large we probably do not want to open the whole file and prepend text, or do some kind of search and replace on text content that size. So my suggestion is using a stream. Also by using a Stream we can avoid entirely writing the file, then having to open the file again and write the text. StreamWriter has a BaseStream property that provides you access to the base stream object which then allows you to set the position of the stream back to the top and write out the information.

Here is my answer.

private void WriteUsers()
{    
    string userCountString = null;
    ASCIIEncoding enc = new ASCIIEncoding();
    byte[] userCountBytes = null;
    int userCounter = 0;
    
    using(StreamWriter sw = File.CreateText("myfile.txt"))
    {
        // Write a blank line and return
        // Note this line will later contain our user count.
        sw.WriteLine();
        
        // Write out the records and keep track of the count 
        for(int i = 1; i < 100; i++)
        {
            sw.WriteLine("User" + i);
            userCounter++;
        }
        
        // Get the base stream and set the position to 0
        sw.BaseStream.Position = 0;
        
        userCountString = "User Count: " + userCounter;
        
        userCountBytes = enc.GetBytes(userCountString);
    
        sw.BaseStream.Write(userCountBytes, 0, userCountBytes.Length);
    }

}

Seattle .net dotnet user group

Great new user group that is getting started and it is in Seattle. Was hosted at Starbucks who provided a top notch facility for the meeting. About 25 people showed up and  speaker Charles Sterling  gave us a great presentation walking us through the new testing tools in Microsoft Visual Studio Team Systems 2010. 

Highly recommend this to anyone involved in .NET development and located in or around Seattle.

http://seattledotnet.org/

They also have a facebook group

http://www.facebook.com/group.php?gid=78132511486

How do I open a file that is in use in C#?

I had a situation recently in which I needed to display in my application the log file that was being generated by the application which was using log4net. Any time I tried to open the log file I would get an exception because the file was in use. I searched google for the answer as I normally do and found many others trying to find the same exact answer. Some came up with ideas like copying the original file to another temporary file and opening it. Others seemed to reach a dead end. However I soon figured out what my issue was.

When reading my file using StreamReader it appears that while I assumed since I was reading the file it should not be an issue if it was in use since I was only reading it. Well that is were I was making my wrong assumption. It appears that the StreamReader object opens the file and tries to set an exclusive lock on the file while reading the contents. Since in my case another program already has an exclusive lock it would fail.

So I started to look at how I could open the file in truly a read only mode. While I found people talking again about opening a file in use for readonly access it was for some reason not clear to me what the exact solution was.

So what is the solution? System.IO.FileShare

By creating a FileStream object and setting FileShare.ReadWrite I am saying that I want to open the file in a mode that allows other files to Read and Write to/from the file while I have it opened.

Here is an example code snipet of how I read my log file that was currently in use by the logging object log4net. Hope this helps someone else.


private void LoadFile()
{
    try
    {
        using(FileStream fileStream = new FileStream(
            "logs/myapp.log",
            FileMode.Open,
            FileAccess.Read,
            FileShare.ReadWrite))
        {
            using(StreamReader streamReader = new StreamReader(fileStream))
            {
                this.textBoxLogs.Text = streamReader.ReadToEnd();
            }
        }
    }
    catch(Exception ex)
    {
        MessageBox.Show("Error loading log file: " + ex.Message);
    }
}


kick it on DotNetKicks.com

Microsoft patterns & practices Application Architecture Guide 2.0

Was listening to a podcast the other day in which one of the authors of the Microsoft patterns & practices Application Architecture Guide was speaking about this latest version so I decide to take a look. So far I am very impressed as the information seems to do a fairly good job of coupling desgin guidlines for applications with some of the latest technologies.

The guide breaks architecture down into 4 parts.

Part I - Fundamentals

  • Fundamentals of Application Architecture
  • .NET Platform Overview
  • Architecture and Design Guidlines

Part II - Desgin

  • Designing Your Architecture
  • Deployment Patterns
  • Architecture Styles
  • Quality Attributes
  • Communication Guidelines

Part III - Layers

  • Layers and Tiers
  • Presentation Layer Guidelines
  • Business Layer Guidelines
  • Data Access Layer Guidelines
  • Service Layer Guidelines

Part IV - Achetypes

  • Application Archetypes
  • Web Applications
  • Rich Internet Applications (RIA)
  • Services
  • Mobile Applications
  • Office Business Applications (OBA)
  • SharePoint LOB Applications

Also the Appendix offers some great advice using a Matrix apporoach to various technologies.

Data Technology Matrix - Provides matrix of old and new technology options such as ADO.NET, Entity Framework, LINQ to SQL and more.

Check out the website on codeplex

Also here is a link to pocket guides based on the Application Architecture Guide but geared towards Web Architecture, Mobile Architecture, RIA Architecture, Rich Client Architecture, Service Architecture.

Check out pocket guides on codeplex

Understanding .NET Reference Types and Value Types

I have been developing in .NET now for about 4 years and while I have visited this topic many times I have not always fully understood things. Dealing with Value Types is really straight forward however when we start to get into Reference Types things become a bit clouded. Well I have recently been forced to try and explain them to someone else and so I decided it was time to make sure I fully understand things. So this is my attempt to provide based on things I have read and learned how things work.

 I will start by simply making some statements that will later help us understand things.

Value Types - Stored on the stack. When you declare a variable based on a value type, the CLR allocates memory for the value type instance wherever the variables is declared.

Reference Types - Variable declared is stored on the stack. When you declare a reference type, the CLR allocates memory for instance on the heap.

Reference Types with Value Types: When declaring inside a reference type a field that is a value type it is important to note that the memory is allocated inside the hosted object.

-----------     ------------
-  stack  -     -  heap    -
-----------     ------------
-  var1   -     -  Class1  -
-----------     - Instance -
-   10    -     ------------
-----------           ^
-  var2   -           |
-----------           |
-   ref   - -----------
-----------

Strings - It is important to point out that strings are immutable. When making changes to a string you are actually creating a new instance of that string. You will see in the examples below that they behave a little different when making changes to them as opposed to making changes to a field in a Class.

Memory Cleanup:

  • Value Types - When a value type goes out of scope the memory is immediatly removed from the stack.
  • Reference Types - When a reference type goes out of scope the variable is immediatly removed from the stack. However the memory allocated on the heap still exists and the removal is managed by the Memory Manager and Garbage Collector process.

 Methods: Passing arguments to methods ByValue and ByReference When you declare a method you by default pass in the argument ByValue. However the behaviour giving this statement is not always as expected when dealing with reference types. Here are some scenarios.

  • Value Type being passed ByValue: Remember that a value type holds the data itself. When passing in ByValue a copy of the actual data is passed. The method now contains its own variable and data and so the result is that the original variable will not change as changes are made to the new variable inside the method.
  • Value Type being passed ByReference: When passing in a value type ByReference we are actually passing a reference is created that points to the data. The result is that any changes made to the variable inside the scope of the method are reflected back to the external variable.
  • Reference Type being passed ByValue: When you pass in a reference type ByValue you get a bit  different behaviour. Remember that the value is always on the heap, so you are actually pointing the new variable to the same memory location as the original variable. Thus when making changes to the variable changes are reflected to the original variable. So if you need to pass a reference type and not have changes reflected you need to make sure you create a full new copy of the object being passed in.
  • Reference Type being passed ByReference: This last situation you most likely want to avoid. This basically ends up creating a reference variable that points to the reference variable that points to the data. So a reference to a reference. While I said you usually should avoid this it is worth pointing out why you might want to use this approach. Remember that in the  previous Reference ByValue changes are reflected back to the original variable. However if you make any changes to the object itself, for example set the object to NULL or set it to a NEW object then the changes would not be reflected outside as your internal variable now points to a new memory location. So if you need to have NEW or NULL changes reflected back outside you could pass it in ByRef.

 Below you will find some examples that will help see how value types and refernce types differ when making changes to them and how the behaviour changes when passing them into methods.

STRING TEST - Reference Type - Immutable
Creating two strings. Setting the first to the value "one-before" and then initializing the second variable by pointing it to the first. Writing out the initial values so that we can see that each is now set to "one-before". So at this point what we have in effect is to variables declared on the stack one and two that both point to the same memory location which holds "one-before". Then we come along and we set one = "one-after" and since strings are immutable a new memory locations is allocated and one is now pointed to the new memory location. However two still remains pointed to the previous memory location so the result is that they now have different values.

//Begin Test
string one = "one-before";
string two = one;

Console.WriteLine(one); //Output = one-before
Console.WriteLine(two); //Output = one-before

one = "one-after";

Console.WriteLine(one); //Output = one-after
Console.WriteLine(two); //Output = one-before
// End Test


INTEGER TEST - Value Type
Creating two integers. Setting the first to the value 1 and then initializing the second variable by pointing it to the first. Writing out the initial values we see that both variables have the same value of 1. The next thing we do is change the value or i = 2. Then we write out the values of the variables and see that i = 2 but j = 1. The reason for this is that value types when assigned a value contain the actual data. So when we created j = i we got a copy of the actual data that i contained and j now has its own data. So when we come along later and make a change to i we are only changing its information and j will still reflect the information it holds.

// Begin Test
int i = 1;
int j = i;

Console.WriteLine(i); // Output = 1
Console.WriteLine(j); // Output = 1

i = 2;

Console.WriteLine(i); // Output = 2
Console.WriteLine(j); // Output = 1
// End Test

To be continued. Will be adding tests for passing into methods Structure (ValueType) and Class (ReferenceType)

 

From Cobalt to BlueQuartz and now BlueOnyx

It has been a bit of a journey since I first started offering web hosting services using the Cobalt Platform that was later sold to Sun Microsystems and then dropped from the product line. However thanks to Sun they released on the cobalt software as opensource and released to the public. Since then a great group of guys dedicated there time to producing an easy to install version based on CentOS called BlueQuartz.

Well it looks like it has now evolved to the next level. BlueOnyx is the latest attempt of keeping this opensource solution alive and going using the latest CentOS 5.2 release offering many new features. Below is some of the verbage from the BlueOnyx site http://www.blueonyx.it

--------------------------------------------------------

BlueOnyx is a Linux distribution (based on CentOS v5.2) which aims at delivering a turnkey server appliance for webhosting. It comes with a web based GUI interface which allows you (and your email, FTP and webhosting clients!) to manage most aspects of the server, its sites and accounts through a browser based interface. It is open source software, released under a Sun modified BSD license.

BlueOnyx is a modernized fork of BlueQuartz and has been developed by Solarspeed Ltd. and NuOnce Networks, Inc. and A lot of improvements and enhancements (compared to a regular BlueQuartz) went into BlueOnyx. Including support for Java Servlet Pages, built in web based email, phpMyAdmin for you and your webhosting clients and provisions to automatically create MySQL databases and users for each virtual site.

Visit BlueOnyx site for more information http://www.blueonyx.it

C# Traceroute using .net framework

If you have ever tried to create a traceroute program using one of the few available ICMP libraries freely available for C# you may have run into some issues mainly to do with the ICMP checksum not being correct. It seems that as of .NET 2.0 framework that microsoft included a Ping class that makes it really easy to then use it to create a traceroute utility. Here is some basic code to create a traceroute utility.

public string Traceroute(string ipAddressOrHostName)
{
    IPAddress ipAddress = Dns.GetHostEntry(ipAddressOrHostName).AddressList[0];
    StringBuilder traceResults = new StringBuilder();
 
    using(Ping pingSender = new Ping())
    {
        PingOptions pingOptions = new PingOptions();
        Stopwatch stopWatch = new Stopwatch();
        byte[] bytes = new byte[32];
 
        pingOptions.DontFragment = true;
        pingOptions.Ttl = 1;
        int maxHops = 30;
 
 
 
        traceResults.AppendLine(
            string.Format(
                "Tracing route to {0} over a maximum of {1} hops:", 
                ipAddress, 
                maxHops));
 
        traceResults.AppendLine();
 
        for(int i = 1; i < maxHops + 1; i++)
        {
            stopWatch.Reset();
            stopWatch.Start();
            PingReply pingReply = pingSender.Send(
                ipAddress, 
                5000, 
                new byte[32], pingOptions);
 
            stopWatch.Stop();
 
            traceResults.AppendLine(
                string.Format("{0}\t{1} ms\t{2}", 
                i, 
                stopWatch.ElapsedMilliseconds, 
                pingReply.Address));
 
            if(pingReply.Status == IPStatus.Success)
            {
                traceResults.AppendLine();
                traceResults.AppendLine("Trace complete."); break;
            }
 
            pingOptions.Ttl++;
        }
 
    }
 
    return traceResults.ToString();
}

Microsoft released free Chart Controls for ASP.NET and Winforms

I might be behind the times but I just found out that MS released free charting controls for Winform and ASP.NET. Requires you be running VS2008.

Charting Component Download : http://www.microsoft.com/downloads/details.aspx?FamilyId=130F7986-BF49-4FE5-9CA8-910AE6EA442C&displaylang=en

Language Pack if you are using other languages: http://www.microsoft.com/downloads/details.aspx?FamilyId=581FF4E3-749F-4454-A5E3-DE4C463143BD&displaylang=en

Visual Studio 2008 Add-On: http://www.microsoft.com/downloads/details.aspx?familyid=1D69CE13-E1E5-4315-825C-F14D33A303E9&displaylang=en

Documentation: http://www.microsoft.com/downloads/details.aspx?familyid=EE8F6F35-B087-4324-9DBA-6DD5E844FD9F&displaylang=en

Also here is a good resouce site that provides samples in asp.net and winforms for using the controls: http://code.msdn.microsoft.com/mschart/Release/ProjectReleases.aspx?ReleaseId=1591

I will be downloading later this evening and playing around with this but this is great to see MS finally release some nice looking chart controls.

c# and vb.net preserving or persisting user settings user.config after upgrades

If you store settings in your application using Settings.Settings and have ever upgraded your application you may have found all the user settings have vanished. It seems that the user.config file is stored in %UserProfile/LocalSettings/Application Data/CompanyName/ApplicationName/versionnumber/user.config

Every time you increment the version on your application a new version folder will be created that is of course empty and the process of populating those user specific settings starts over again.

Here is a way you can use to persist those settings after an upgrade.

C# Directions:

In your project open Settings.Settings and add a new user setting called ShouldUpgrade and set this to be a boolean value and set the value to true. Now click View Code icon at the top left side of the settings window. In the constructor add something like this.

public Settings()
{
    if(this.ShouldUpgrade)
    {
       
this.Upgrade();
       
this.ShouldUpgrade = false;
       
this.Save();
    }
}

So the first time that the settings class is instantiated it will check the ShouldUpgrade setting of the new user.config file which will be set to true. The this.Upgrade() then copies any user settings from the most recent version prior to this upgrade. Then set ShouldUpgrade to false and save the settings.

Also because we stored ShouldUpgrade as a user setting this upgrade will happen for each individual user on the same machine.

For VB.NET you do the same approach but use My.Settings.Upgrade()