Sunday, March 06, 2011

T4 Template Tour of First Gear

At the Boise Code Camp, I lead off with a demonstration of some basic features of T4 Templates. The point of the presentation is to get you from no T4 to the level of most blog examples. I got good reviews from the few who attended my session, so I thought I would attempt to write it down. Here goes…

The Script

First, create a Text file and change the extension to "tt". We’ll call it "Hello.tt". I will add a template directive and tell T4 that I want to use C#. I add the text "Hello World" below the directive. When I save it, "Hello.cs" appears below the template.

Template (Hello.tt):
<#@ template language="C#" #>
Hello World
Output (Hello.cs):
Hello World

Also notice that Visual Studio complains that Hello.cs won’t compile. For this demonstration, I don’t want to create a C# code file, so I will tell the template to create a "txt" file with an output directive.

Template (Hello.tt):
<#@ template language="C#" #>
<#@ output extension="txt" #>
Hello World
Output (Hello.txt):
Hello World

Visual Studio is happy because it doesn’t try to compile "txt" files. (Since it doesn't change, I won't repeat the output again.)

Now, let’s add some actual C# template code, I’m going to create a variable called "name" in a Statement Block and reference it in an Expression Block.

Template (Hello.tt):
<#@ template language="C#" #>
<#@ output extension="txt" #>
<# string name = "World"; #>
Hello <#= name #>

Now I’m going to replace the variable with a function. The function will need to be in a Class Feature Block. Just to speed things up, I’m going to use a Write call in a Statement Block to render the name. You can write text to the output using either Expression Blocks OR calls to Write().

Template (Hello.tt):
<#@ template language="C#" #>
<#@ output extension="txt" #>
Hello <# Write(GetName()); #>
<#+ 
    string getName()
    {
        return "Word";
    }
#>

Now I want to move my function into an include file. I will create a new text file and call it "HelloFunctions.tt" I move the Class Feature Block into that file. Notice that there is no template directive. Also, you must save the include file BEFORE you save the template file.

Include (HelloFunctions.tt)
<#+ 
    string getName()
    {
        return "Word";
    }
#>
Template (Hello.tt):
<#@ template language="C#" #>
<#@ include file="HelloFunctions.tt" #>
<#@ output extension="txt" #>
Hello <# Write(GetName()); #>

Now, for a brief look under the hood, I can show you the source code for the Text Transformation. Here I add "debug="true"" to the template file, save it and look in the temp directory (you can get your temp directory by typing "set temp" in a command prompt)

Template (Hello.tt):
<#@ template language="C#" #>
<#@ include file="HelloFunctions.tt" #>
<#@ output extension="txt" #>
Hello <# 
// This comment will show up in .cs file
Write(GetName()); 
#>
Generated C# File (hcs3svyj.0.cs in this case)
namespace Microsoft.VisualStudio.TextTemplatingBFD11436D0A4595408B06EA4E1B2A636 {
    using System;
    using Microsoft.VisualStudio.TextTemplating.VSHost;

    
    #line 1 "c:\vsprojects\SimpleT4\Hello.tt"
    public class GeneratedTextTransformation : 
        Microsoft.VisualStudio.TextTemplating.TextTransformation {
        public override string TransformText() {
            try {
                this.Write("Hello ");

                #line 4 "c:\vsprojects\SimpleT4\Hello.tt"

// This comment will show up in .cs file
Write(getName()); 

          
                #line default
                #line hidden
            }
            catch (System.Exception e) {
                System.CodeDom.Compiler.CompilerError error = new 
                    System.CodeDom.Compiler.CompilerError();
                error.ErrorText = e.ToString();
                error.FileName = "c:\\vsprojects\\SimpleT4\\Hello.tt";
                this.Errors.Add(error);
            }
            return this.GenerationEnvironment.ToString();
        }

        #line 1 "c:\vsprojects\SimpleT4\HelloFunctions.tt"

 string getName()
 {
  return "World";
 }

  
        #line default
        #line hidden
    }

    #line default
    #line hidden
}

These source files have helped me understand what is going on behind the scenes. For example, I didn’t really get the difference between Statement Blocks (<# #>) and Class Feature Blocks (<#+ #>) until I saw the generated.

Afterthought

This isn’t a transcription! Written and spoken communication is different by nature.

In the presentation I would change the value of "World" with different words ("Earth", "Boise", "Boise Code Camp", etc); In a presentation, this added proofiness that the templates were run and some lame humor to lighten the mood of the talk. In the written form, it would force me to repeat the Output after each sample (and make this longer).

Tuesday, February 22, 2011

Boise Code Camp: Code Generation with T4 Templates

I will be presenting at the Boise Code Camp on Saturday, February 26, 2011, 9:30 AM in the Farnsworth room in the BSU SUB!

T4 Templates (Text Template Transformation Toolkit) is the code generator built into Visual Studio. This session is a basic introduction to code generation in C# and .NET using T4 Templates in Visual Studio 2008. I will takes you on a journey starting from the basic "Hello World" template into a world where you can wrote reusable templates and interrogate SQL Server or an ADO.NET DataSet for table descriptions.

Check it out!

Wednesday, February 16, 2011

Four Styles of T4 Template Code

As I work through T4 templates as I get ready for Boise Code Camp, I’ve noticed a few styles of template code. Just for kicks, I am going to name them and give an example of each using a code generation that I blogged about early last year. Each template uses an include file that I present after the four samples. Each template relies on its style to the point of absurdity; I didn’t go as far as to eliminate all text blocks.

Real life templates are mixtures of these styles applied at appropriate places.

I adapted the code generation that I did here

Embed

This can also be called the Classic ASP style. All generated code is added as text blocks or expression blocks (<#= #>). The logic is contained in code blocks mixed in with the text and expression blocks. For simple logic, this style is probably OK. It can be difficult to read and maintain as the logic gets more complex. This is the most concrete approach.

<#@ template language="C#v3.5" debug="true" #>
<#@ output extension="sql" #>
<#@ include file="TableDescriptionUtilCS.tt" #>
<#
    string ConnectionString = @"Data Source=localhost\SQLEXPRESS;" + 
        "Initial Catalog=Northwind;Integrated Security=True";
    string Table = "Employees"; 

    TableDescriptionUtil tdu = new TableDescriptionUtil();
    List<TableDescription> Columns = tdu.GetTableDescription(ConnectionString, Table); 

    int counter;
#>
CREATE PROCEDURE [dbo].[Insert<#=Table #>]
<#
    counter = 1;
    foreach (TableDescription item in Columns)
    {
        #>    @<#= item.ColName #> <#= item.FullTypeName #><#
        if (item.Status == 0x80)
            #> OUT<#
        if (item.ColIsNullable != 0) 
            #> = NULL<#
        if (counter < Columns.Count)
            #>,<#
        counter++;
        #><#= "\r\n"#><#
    }
#>

AS

INSERT INTO [<#=Table #>] (<#
    counter = 1;
    foreach (TableDescription item in Columns)
    {
        if (item.Status != 0x80)
        {
            if (counter > 1)
                #>, <#
            if (counter % 5 == 0)
                #><#= "\r\n    " #><#
            counter++;
            #>[<#= item.ColName #>]<#
        }
    }
#>)
VALUES (<#
    counter = 1;
    foreach (TableDescription item in Columns)
    {
        if (item.Status != 0x80)
        {
            if (counter > 1)
                #>, <#
            if (counter % 5 == 0)
                #><#= "\r\n    " #><#
            counter++;
            #>@<#= item.ColName #><#
        }
    }
#>)

SET @<#
    foreach (TableDescription item in Columns)
    {
        if (item.Status == 0x80)
        {
            #><#=item.ColName #><#
            break;
        }
    }
#> = @@IDENTITY

Inline Write

This style uses code blocks that contain Write(), WriteLine(), PushIndent(), PopIndent(), and other statements.

This approach is more abstract that Embed, it is easier for me to read than Embed or String Function.

<#@ template language="C#v3.5" debug="true" #>
<#@ output extension="sql" #>
<#@ include file="TableDescriptionUtilCS.tt" #>
<# 
    string ConnectionString = @"Data Source=localhost\SQLEXPRESS;" +
        Initial Catalog=Northwind;Integrated Security=True";
    string Table = "Employees";

    TableDescriptionUtil tdu = new TableDescriptionUtil();
    List<TableDescription> Columns = tdu.GetTableDescription(ConnectionString, Table);

    int counter;
#>
CREATE PROCEDURE [dbo].[Insert<# Write(Table); #>]
<#
    PushIndent("    ");
    counter = 1;
    foreach (TableDescription item in Columns)
    {
        Write("@" + item.ColName + " " + item.FullTypeName);
        if (item.Status == 0x80)
            Write(" OUT");
        if (item.ColIsNullable != 0)
            Write(" = NULL");
        if (counter < Columns.Count)
            WriteLine(",");
        counter++;
    }
    PopIndent();
#>


AS

INSERT INTO [<# Write(Table); #>] (<#
    PushIndent("    ");
    counter = 1;
    foreach (TableDescription item in Columns)
    {
        if (item.Status != 0x80)
        {
            if (counter > 1)
                Write(", ");
            if (counter % 5 == 0)
                WriteLine(String.Empty);
            counter++;
            Write(string.Format("[{0}]", item.ColName));
        }
    }
    PopIndent();
#>)
VALUES (<#
    PushIndent("    ");
    counter = 1;
    foreach (TableDescription item in Columns)
    {
        if (item.Status != 0x80)
        {
            if (counter > 1)
                Write(", ");
            if (counter % 5 == 0)
                WriteLine(string.Empty);
            counter++;
            Write(String.Format("@{0}", item.ColName));
        }
    }
    PopIndent();
#>)

SET @<#
    foreach (TableDescription item in Columns)
    {
        if (item.Status == 0x80)
        {
            Write(item.ColName);
            break;
        }
    }
#> = @@IDENTITY

String Function

Here I call a function in an expression block that returns a bit of generated code as a string. The functions are in feature blocks (or include file, linked assemblies, etc.).

Since I did some primitive string based code gen for a number of years, I find myself relying on this approach. This approach can lead to code just as ugly as the Embed approach. This approach is good for adapting old fashioned code gen to T4.

<#@ template language="C#v3.5" debug="true" #>
<#@ output extension="sql" #>
<#@ include file="TableDescriptionUtilCS.tt" #>
<# 
    string ConnectionString = @"Data Source=localhost\SQLEXPRESS;" + 
        "Initial Catalog=Northwind;Integrated Security=True";
    string Table = "Employees";

    TableDescriptionUtil tdu = new TableDescriptionUtil();
     List<TableDescription> Columns = tdu.GetTableDescription(ConnectionString, Table);

     int counter;
#>
CREATE PROCEDURE [dbo].[Insert<#= getTableName(Table) #>]
<#= getParameterList(Columns) #>

AS

INSERT INTO [<#= getTableName(Table) #>] (<#= getFieldNames(Columns) #>)
VALUES (<#= getValueNames(Columns) #>)


SET @<#=getIdentityFieldName(Columns) #> = @@IDENTITY
<#+
    string getTableName(string table)
    {
        // Yes, this is hard core overkill
        return table;
    }
    string getParameterList(List<TableDescription> columns)
    {
        string rVal = string.Empty;
        int counter = 1;
        foreach (TableDescription item in columns)
        {
           rVal += "    @" + item.ColName + " " + item.FullTypeName;
           if (item.Status == 0x80)
               rVal += " OUT";
           if (item.ColIsNullable != 0)
               rVal += " = NULL";
           if (counter < columns.Count)
               rVal += ",\r\n";
           counter++;
       }
       return rVal;
    }
    string getFieldNames(List<TableDescription> columns)
    {
        string rVal = string.Empty;
        int counter = 1;
        foreach (TableDescription item in columns)
        {
            if (item.Status != 0x80)
            {
                if (counter > 1)
                    rVal += ", ";
                if (counter % 5 == 0)
                    rVal += "\r\n    ";
                counter++;
                rVal += string.Format("[{0}]", item.ColName);
           }
        }
        return rVal;
    }
    string getValueNames(List<TableDescription> columns)
    {
        string rVal = string.Empty;
        int counter = 1;
        foreach (TableDescription item in columns)
        {
        if (item.Status != 0x80)
        {
            if (counter > 1)
                rVal += ", ";
            if (counter % 5 == 0)
                rVal += "\r\n    ";
            counter++;
            rVal += string.Format("@{0}", item.ColName);
        }
    }
    return rVal;
    }
    string getIdentityFieldName(List<TableDescription> columns)
    {
        string rVal = string.Empty;
        foreach (TableDescription item in columns)
        {
            if (item.Status == 0x80)
            {
                rVal =item.ColName;
                break;
            }
        }
        return rVal;
    }
#>

Write Function

Here I call a function to generate code a block of code with Write functions.

Unlike the String Function approach, the code generation is relying on a side effect of the function, so this approach isn’t functional.

<#@ template language="C#v3.5" debug="true" #>
<#@ output extension="sql" #>
<#@ include file="TableDescriptionUtilCS.tt" #>
<# 
    string ConnectionString = @"Data Source=localhost\SQLEXPRESS;" + 
        "Initial Catalog=Northwind;Integrated Security=True";
    string Table = "Employees";

    TableDescriptionUtil tdu = new TableDescriptionUtil();
    List<TableDescription> Columns = tdu.GetTableDescription(ConnectionString, Table);

    int counter;
#>
CREATE PROCEDURE [dbo].[Insert<# writeTableName(Table); #>]
<# writeParameterList(Columns); #>

AS

INSERT INTO [<# writeTableName(Table); #>] (<# writeFieldNames(Columns); #>)
VALUES (<# writeValueNames(Columns); #>)


SET @<# writeIdentityFieldName(Columns); #>  = @@IDENTITY
<#+
    void writeTableName(string table)
    {
        Write(table);
    }
    void  writeParameterList(List<TableDescription> columns)
    {
        PushIndent("    ");
        int counter = 1;
        foreach (TableDescription item in columns)
        {
            Write("@" + item.ColName + " " + item.FullTypeName);
            if (item.Status == 0x80)
                Write(" OUT");
            if (item.ColIsNullable != 0)
                Write(" = NULL");
            if (counter < columns.Count)
                WriteLine(",");
            counter++;
        }
        PopIndent();
    }
    void writeFieldNames(List<TableDescription> columns)
    {
         PushIndent("    ");
         int counter = 1;
         foreach (TableDescription item in columns)
         {
             if (item.Status != 0x80)
             {
                 if (counter > 1)
                     Write(", ");
                 if (counter % 5 == 0)
                     WriteLine(String.Empty);
                 counter++;
                 Write(string.Format("[{0}]", item.ColName));
             }
         }
         PopIndent();
    }
    void writeValueNames(List<TableDescription> columns)
    {
        PushIndent("    ");
        int counter = 1;
        foreach (TableDescription item in columns)
        {
            if (item.Status != 0x80)
            {
                if (counter > 1)
                    Write(", ");
                if (counter % 5 == 0)
                    WriteLine(string.Empty);
                counter++;
                Write(String.Format("@{0}", item.ColName));
            }
        }
        PopIndent();
    }
    void writeIdentityFieldName(List<TableDescription> columns)
    {
        foreach (TableDescription item in columns)
        {
            if (item.Status == 0x80)
            {
                Write(item.ColName);
                break;
            }
        }
    }
#>

Include File: TableDescriptionUtil.tt

I included this file in all the examples above.

<#@ assembly name="System.Data.dll" #>
<#@ import namespace="System.Data.SqlClient" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ import namespace="Microsoft.VisualStudio.TextTemplating" #>
<#+
public class TableDescriptionUtil : TextTransformation
{
    /// <remarks>
    /// I don't actually call this, but it is necessary for T4 Templates to work
    /// </remarks>
    public override string TransformText()
    {
        return "";
    }
    /// <summary>
    /// Gets a DataReader containing selected column information for all of the
    /// columns in the indecated table from SQL Server System Tables
    /// </summary>
    /// <param name="tableName">Get information on THIS table</param>
    /// <param name="conn">An existing open connection to be used </param>
    /// <returns>A DataReader pointing to the results</returns>
    private SqlDataReader GetTableDefReader(string tableName , SqlConnection conn )
    {
        SqlDataReader reader;// = new SqlDataReader();
        string getDataForTable = 
             " SELECT c.name AS col_name, " +
             "   t.name AS type_name, " +
             "   CAST(c.length AS INT) AS length, " +
             "   c.isnullable, " +
             "   CAST(c.status AS INT) AS status " +
             " FROM " +
             "  sysObjects o " +
             "  JOIN sysColumns c ON c.id = o.id " +
             "  JOIN systypes t on c.xtype = t.xtype AND t.status = 0 " +
             " WHERE o.name = @tableName " +
             " ORDER BY c.colid ";
        using (var cmd = new SqlCommand(getDataForTable, conn))
        {
            cmd.Parameters.AddWithValue("@tableName", tableName);
            reader = cmd.ExecuteReader();
        }
        return reader;
    }

    /// <summary>
    /// Gets a List of column descriptions for a given table
    /// </summary>
    /// <param name="ConnectionString">Connection string to the database</param>
    /// <param name="Table">The name of the table to describe</param>
    /// <returns>A list of column descriptions</returns>
    public List<TableDescription> GetTableDescription(string ConnectionString, 
         string Table) 
    {
        SqlDataReader reader;
        List<TableDescription> Columns = new List<TableDescription>();
        using (var conn = new SqlConnection(ConnectionString))
        {
            conn.Open();
            reader = GetTableDefReader(Table, conn);
            while (reader.Read())
            {
                TableDescription Column = new TableDescription();
                Column.ColName = reader.GetString(0);
                Column.TypeName = reader.GetString(1);
                Column.ColLength = reader.GetInt32(2);
                Column.ColIsNullable = reader.GetInt32(3);
                Column.Status = reader.GetInt32(4);
                Column.FullTypeName = GetTypeName(Column.TypeName, Column.ColLength);
                Columns.Add(Column);
            }
            conn.Close();
        }
        return Columns;
    }
    /// <summary>
    /// Gets the TypeName as it should appear in a SP Parameter List
    /// </summary>
    /// <param name="TypeName"></param>
    /// <param name="ColLength"></param>
    /// <returns>The processed Type Name as a string</returns>
    private string GetTypeName(string typeName , int ColLength )
    {
        switch (typeName.ToLower())
        {
        case "varchar":
        case "nvarchar":
            string lenString  = ColLength.ToString();
            if (ColLength == -1 )
                lenString = "MAX";
            return string.Format("{0}({1})", typeName, lenString);
        default:
            return typeName;
        }
    }
}
public class TableDescription
{
    public string ColName {get; set;}
    public string TypeName {get; set;}
    public string FullTypeName {get; set;}
    public int ColLength {get; set;}
    public int Status {get; set;}
    public int ColIsNullable {get; set;}
}
#>

Wednesday, February 02, 2011

Happy Groundhog Day!

Traditionally we have pork sausage for dinner. I leave it as an exercise to the reader to work out the bad pun.

Sunday, January 23, 2011

Referencing Assemblies: T4 Gotcha

When I run T4 Templates through Visual Studio 2008, I don't have to give the full path to reference GAC assemblies:

<#@ assembly name="System.Data.dll" #>

But when I run the template in my own executable using Microsoft.VisualStudio.TextTemplating.Engine, I have problems.

If the Template code is in VB.NET, I get:

vbc : Command line (0,0) : error BC2006: Compiling transformation: option 'r' requires ': <file_list>'

If the Template code is in C#, I get:

c:\Users\jacks\AppData\Local\Temp\rnftuopt.0.cs(3,18) : error CS0234: Compiling transformation: The type or namespace name 'Data' does not exist in the namespace 'System' (are you missing an assembly reference?)

To fix this, I referenced the full path of the assembly.

<#@ assembly name="C:\Windows\Microsoft.NET\Framework\v2.0.50727\System.Data.dll" #>

NOTE: This doesn't seam to apply to non-GAC assemblies; I have noticed that I don't have give the full path of my own assemblies as long as they are in the same folder as my executable.

Friday, January 14, 2011

Fulfilled my contract with AT&T

Just over a two years ago, I bought an iPhone 3G (no S).

In Spokane, I haven't have any problems AT&T. But Spokane is the first place named in AT&T's Postcard Commercial in 2009, so AT&T knows that we get better than average service here in Spocompton.

If I was to stay in Spokane, I would probably be safe getting an upgrade from AT&T. The trouble is that I am working month to month as a temp and I don't know where I will be working if/when my current contract runs out.

I've hear horror stories of bad service in real cities like New York or Chicago. And Verizon will offer the iPhone 4 next month; some say that AT&T's problems are caused by the popularity of the iPhone and Verizon will degrade too.

I think I will stay with my antique phone for awhile.

Wednesday, January 05, 2011

Running FTP.EXE from C#

NOTE: Ultimately, I didn't go with this solution.

Since I need to log on to various FTP servers with various security configurations and FTP.EXE seams to figure out these configurations and the free FTP libraries I’ve played with can’t I have decided to wrap FTP.EXE with C#.

Telling it what to do

I wanted to avoid sending my commands to FTP.EXE via command files because I would have to dynamically write text files and make sure that they are secure. System.Diagnostics.Process allows me to send commands to its StandardInput stream, but I can’t figure out how to make it accept passwords. To solve these problems, I log in using command files (which I can secure and only allow myself and the program to see, and then write the rest of the commands to the StandardInput stream.

The Command File

The command file needs to reside in a directory with no spaces and the file needs to be saved using ANSI encoding. Here I open a server and login, the rest is up to the commands that I send to the stream.

open localhost
username
password

Sample Application

This simple console application demonstrates using the command file to log in and StandardInput to send a List command (and Quit).

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;

namespace DemoFtp
{
    class Program
    {
        private const string FTP_EXE_LOCATION = @"C:\Windows\System32\ftp.exe";
        private const string DEFAULT_COMMAND_FILE_LOCATION = @"C:\temp\localhost.txt";

        static void Main(string[] args)
        {
            try
            {
                string commandFileLocation = getCommandFileLocation(args);

                var commands = new List<string>();
                commands.Add("ls -la");
                commands.Add("quit");   // Don't forget to log out!
                string output = RunWithResponses(
                    FTP_EXE_LOCATION, 
                    string.Format("-s:{0}", commandFileLocation), 
                    commands);
                Console.WriteLine(output);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            Console.ReadKey();
        }

        private static string getCommandFileLocation(string[] args)
        {
            string commandFileLocation;
            if (args.Length > 0)
                commandFileLocation = args[0];
            else
                commandFileLocation = DEFAULT_COMMAND_FILE_LOCATION;
            return commandFileLocation;
        }

        public static string RunWithResponses(string exeName, string argsLine,
            List<string> commands)
        {
            StreamReader outputStream = StreamReader.Null;
            StreamReader errorStream = StreamReader.Null;
            StreamWriter inputStream = StreamWriter.Null;
            string output = string.Empty;
            try
            {
                // Setup the process
                Process newProcess = new Process();
                newProcess.StartInfo.FileName = exeName;
                newProcess.StartInfo.Arguments = argsLine;
                newProcess.StartInfo.UseShellExecute = false;
                newProcess.StartInfo.CreateNoWindow = true;
                newProcess.StartInfo.RedirectStandardInput = true;
                newProcess.StartInfo.RedirectStandardOutput = true;
                newProcess.StartInfo.RedirectStandardError = true;
                newProcess.Start();
                inputStream = newProcess.StandardInput;
                outputStream = newProcess.StandardOutput;
                errorStream = newProcess.StandardError;
                inputStream.AutoFlush = true;

                // Issue My commands
                foreach (string command in commands)
                {
                    inputStream.WriteLine(command);
                }

                // Get the rsults and wait for it to end
                output = outputStream.ReadToEnd();
                newProcess.WaitForExit();
            }
            finally
            {
                // Clean up
                if (outputStream != StreamReader.Null)
                    outputStream.Close();
                if (errorStream != StreamReader.Null)
                    errorStream.Close();
                if (inputStream != StreamWriter.Null)
                    inputStream.Close();
            }
            return output;
        }
    }
}