Peter Seale's weblog

Monday, August 24, 2009 10:08:21 PM UTC 

The PowerShell REPL is awesome.

PowerShell is by no means the only REPL. There's the immediate window in Visual Studio, the Snippet Compiler, LINQPad, the Interactive C# shell from Mono, and a REPL environment for most every other scripting language on the planet. Some of the TDD guys refer to "exploratory tests" that they write to learn about a third-party API. On the Regex front, there are scads of web-based and Windows-based tools to help you build and test regular expressions as fast as you can hit the "Run" button. I'll even accept writing a console application as a weak form of a REPL, though I wouldn't encourage it. All these things serve the same goal: give me instantaneous feedback. For those of you already familiar with the REPL, we're good, we're in the know.

But if you're the person who never uses a REPL, allow me to show you, using an example from just 3 minutes ago, how powerful they are.

My burning question

All this began with a burning question: what happens in string.Format() if I place the parameters out of order? What happens if I use a parameter twice?

The question, answered within 1 minute using the PowerShell REPL

Conclusion stated in words

I answered a specific question about .NET's string.Format() library function in less time than it would have taken to search and peruse the search results. Sandboxes such as these reduce the friction and enable me to run a series of experiments as quickly as I can think of them. Good REPLs (like PowerShell) allow me to a) quickly get feedback on my input commands, b) format and parse the resulting objects into a meaningful answer. Bad feedback loops (things that aren't REPLs) require overhead to even run, deliver feedback in over an hour or even as late as the next day or the next week, and deliver meaningless answers (think huge log files). I'm just here to make sure you're all aware: you have a choice: you can choose a REPL, or you can choose awfulness. Your call.

Oh yeah, what's a REPL?

Read-eval-print loop

Tuesday, October 28, 2008 11:46:47 PM UTC 
I couldn't hold out.

It's okay though, because today we're strictly business. In the course of developing a bunch of SharePoint timer jobs recently, I've learned several things, most of which aren't obvious from the get-go:

1. Storing and retrieving configuration data is a problem. Because I don't have a farm-wide configuration list (yet—the temptation grows every day), I was forced to do some ugliness in order to store and retrieve configuration data. I don't necessarily recommend my approach; instead I'll just say I'm using a custom SPPersistedObject as my Timer Job's config store and I'll further say that it works, roughly, though I'd now prefer a better way. Consider setting up a farm-wide config list, it's a relatively big time investment but is probably worth it. Other traditional config storage options (such as the web.config, or a site-local config list, or your site-local property bag) aren't accessible to Timer Jobs without some sort of…configuration's configuration…hmm, yes…without something extra pointing the way. Anyway, it's a problem.
   
2. ANYTIME YOU UPDATE YOUR TIMER JOB CLASS (or custom assembly), YOU MUST BUMP ALL SHAREPOINT TIMER SERVICES ON THE FARM! I learned this lesson the hard way. If you fail to bump the timer service, it will blissfully run the old copy of your timer job class; I don't know how or why it caches your assembly, but it does, and bumping the SharePoint timer service is the only way to clear the assembly cache(?) and force it to use your minty fresh assembly.
1. Side-note: can we report this as a bug in the Solution framework? Because this is a big enough gotcha that the Solution framework needs to include an option to -bumptimersvc …or something. Maybe a custom stsadm command (stsadm -o resetadmsvc) or maybe tack something onto the Solution deployment API and associated stsadm commands…we need something.
   
3. There are differences between the context of a test harness (i.e. something like an NUnit integration test running under the NUnit test runner) and a timer job running in the Timer service. This may sound obvious, but when you're troubleshooting something that "only breaks on the test farm," this little bit of trivia is important. If you need to troubleshoot your timer job as it runs on the timer service, specifically, this can get tricky.
   

Also, in the course of troubleshooting just such an issue (as outlined in #3) I've created a little script to speed up the code-compile-test loop; instead of scheduling a timer job for "0100 hours" and waiting until tomorrow to see the results, why not reschedule the timer job by your own self? And that's exactly what I did.

My script below will reschedule your timer job to run 10 seconds in the future (read: instantaneously). All you have to do to get this script working for you is customize four variables to match your own timer job, then follow the quick "usage instructions" at the bottom of the script. Below is a rundown of the four variables:

• $siteUrl - the site collection root URL. We need this to get a reference to the SPWebApplication that holds your Timer Job.
• $customAssemblyName - the partial name of your custom assembly. This is necessary because we're going to new up an instance of your timer job, and thus we'll need to first load the containing assembly.
  
• $jobName - this need only be a rough equivalent of your job name. I'm usually lazy and say something like "*custom profile job*" or the minimum necessary to identify my job from all the rest. Messy is good; we're running a one-off script, right? Or you can go ahead and type the perfect exact case-sensitive job name in there, that's fine too.
• $timerJobClassName - again, we need this because we're going to new up a rescheduled timer job.
  

Assumptions:

• Your original schedule doesn't matter, and may be destroyed. Because that's exactly what this script does, by the way—it destroys the original schedule and sets a "10 seconds from now" schedule. Incidentally, whatever it did before, your job now runs on a daily schedule :)
  
• You're only concerned about the timer job running on one web application's context. Because in truth that's all that mattered to me when I wrote this script, I didn't consider the possibility of multiple jobs.
• You're running a single-server farm (i.e. a developer VM). My script only stops the service on the local server.
  
• No one else cares if you bump the SPTimerV3 service, including any other timer jobs that may be running presently. Note in the script below, PowerShell has some cmdlets to work with Windows Services. I was totally unaware of them until I had to bump this service; neat.
  

While these assumptions sound scary, trust me—you won't care. On a single developer VM, you won't care about all these things. Even on a multi-server test farm, you won't care—because this script is going to save you hours of troubleshooting.

The PowerShell script is as follows:

$siteUrl = "http://dev"
$customAssemblyName = "Corp.SharePoint.Assembly"
$jobName = "*your job name*wildcards*work*"
$timerJobClassName = "Corp.SharePoint.Namespace.TimerJob"

[void][reflection.assembly]::LoadWithPartialName("Microsoft.SharePoint")
[void][reflection.assembly]::LoadwithPartialName("Microsoft.Office.Server")
[void][reflection.assembly]::LoadwithPartialName($customAssemblyName)

function Run-Init
{
    $global:s = [Microsoft.SharePoint.SPSite]$siteUrl
    $global:webApplication = $s.WebApplication
    $global:job = $webApplication.JobDefinitions | ? { $_.Name -like $jobName }
}

function Create-NewJob
{
    Stop-Service "SPTimerV3"
    Start-Service "SPTimerV3"
    $global:job.Delete()
    $global:job = new-object $timerJobClassName -arg $webApplication
    $sched = new-object Microsoft.SharePoint.SPDailySchedule
    $now = [datetime]::now.AddSeconds(10)
    $sched.BeginHour = $now.Hour
    $sched.EndHour = $now.Hour
    $sched.BeginMinute = $now.Minute
    $sched.EndMinute = $now.Minute
    $sched.beginsecond = $now.Second
    $sched.endsecond = $now.Second
    $global:job.Schedule = $sched
    $global:job.Update()
}

#Usage: paste this script directly into a PowerShell console; the quickest
#way is to right-mouse-button click. Then when you're ready,
#run the following commands (minus the # of course):
#
#Run-Init
#Create-NewJob
#
#Anytime you update your custom assembly "Corp.SharePoint.Assembly", you will need to
#DESTROY your open PowerShell console/session and create a new one. This is the cleanest way
#to unload your old custom assembly.

That's pretty much it. Change the variables to whatever you need, open a PowerShell console, right-click, then type "Run-Init; Create-NewJob". You're done! Step 3: Profit!

Tiny footnote: if you don't care about "context", this script also allows you to execute the timer job immediately. First run the "Run-Init" function, then just type $job.Execute([guid]::Empty) in PowerShell. You can also attach to the PowerShell.exe process and do "remote debugging" of your timer job, if desired. Though if you're going to go that far, you should probably just write an NUnit test that performs the same task, and debug THAT. I'm very pro-unit testing frameworks, really, they're great. Anything that closes the code-compile-test loop, in any way, is a good thing.

Tuesday, October 28, 2008 1:52:14 AM UTC 

Yes, I'm aware it's late in the year 2008, I'm aware this stuff isn't as fresh as WPF 3D or Ruby Processing.

As I've posted earlier, I've accrued some treasured junk. Now that I have all this junk, what am I to do? Well, um…I didn't really know either.

So I started messing around.

Messing around with System.Drawing: first, infrastructure

The first thing I did was to determine the average color for a single image. I'm not sure exactly where I'm going, but I figure, hey, if you want to get a rough "picture" of what an image looks like, it's not a bad idea to look at the average color value. And we're using the RGB breakdown for color, meaning white is #FFFFFF (256,256,256), black is #000000 (0,0,0), and everything else falls in between.

Note that in my case, performance is not a big deal; I'm doing all these calculations one pixel at a time which, as you might image, is suboptimal. Mostly a straightforward operation:

public static Color Average(Image image)
{
    using (Bitmap bitmap = new Bitmap(image))
    {
        int red, green, blue;
        long redRunningSum = 0, greenRunningSum = 0, blueRunningSum = 0;
        long numPixels = bitmap.Width * bitmap.Height;

        foreach (Color pixelColor in ImageHelper.GetPixelsFor(bitmap))
        {
            redRunningSum += pixelColor.R;
            blueRunningSum += pixelColor.B;
            greenRunningSum += pixelColor.G;
        }

        red = (int)(redRunningSum / numPixels);
        green = (int)(greenRunningSum / numPixels);
        blue = (int)(blueRunningSum / numPixels);

        return Color.FromArgb(red, green, blue);
    }
}

Ok, so why do we care—it's a function, right? Well, okay, yes—but here's a PowerShell function you may also find interesting:

function Average-Images ($filenames)
{
    [void][reflection.assembly]::Loadfile("C:\a\sandbox\ImgTest\bin\Debug\ImgTest.dll")
    $i = 1
    $total = $filenames.count
    $results = @()
    foreach ($filename in $filenames)
    {
        write-host "$i - $($i*100/$total)%- $($filename)"
        $i++
        $img = [System.Drawing.Image]::FromFile($filename)
        $o = new-object PSObject
        $avg = [ImgTest.ImageHelper]::Average($img)
        add-member -inp $o -membertype "NoteProperty" -name "Filename" -value $filename
        add-member -inp $o -membertype "NoteProperty" -name "Image" -value $img
        add-member -inp $o -membertype "NoteProperty" -name "Red" -value $avg.R
        add-member -inp $o -membertype "NoteProperty" -name "Green" -value $avg.G
        add-member -inp $o -membertype "NoteProperty" -name "Blue" -value $avg.B
        $results += $o
    }
    $results
}

So. This is getting interesting. What the "Average-Images" function above does is create a custom object with some useful properties: we've got the original filename, we've got a still-breathing reference to the System.Drawing.Image object, and we're storing the "average pixel's" red, green, blue values as individual properties. The resulting objects look something like this:

Maybe it's still not interesting for you. That's fine, 'cause this party's* just getting started!
*despite what I've just written, this is not a party

I have one more piece of "infrastructure" to explain, before we can get cooking: I've created a PowerShell function called "Make-Html," which creates a permanent HTML file listing all the images I want to see, in the order I want to see them. As an added bonus, the function immediately launches the newly-created file in my browser. Here's the code:

$startDir = "C:\a\ps1\scrape\"
function Make-Html ($fullfilenames, $resultingFilename)
{
    $files = $fullFilenames | % { $_.split("\")[-1] }
    $tags = $files | % { "<div style=""float:left;""><img src=""$_""/></div>" }
    $html = @"
<html><head><title>$($resultingFilename)</title>
</head><body>
$($tags)
</body></html>
"@

    $html > "$($startDir)$($resultingFilename).html"
    ii "$($startDir)$($resultingFilename).html"
}

Ok, I know, we're still not doing anything.

Let's warm up

Okay, as I say to everyone, the real power of PowerShell is its object piping. PowerShell pipes objects, not text; this is something best seen, not heard, and hopefully we'll see a little something today. The objects we'll be slinging through the pipeline today are, as mentioned above, custom objects that have a Filename, an Image, and the RGB values representing the image's average (mean?) color.

So, let's count how many items we have:

Awesome. Let's count how many items we have that are more red than any other color:

Hmm, that was unexpected, 359 red-dominant images out of 503, that's proportionally huge. I'll point out that I did some extra fanciness to get this count to evaluate on one line, but usually (i.e. when I'm not posting to my blog) I'll work my way in parts, not all at once. So the same thing, split out, would be:

That's more realistic.

Okay, one more thing before we go. Finding out most of my pictures are red-dominant has me wondering: what about the other two? Let's work with the objects a little* to massage the answer out of them:
*a lot; ugly function that pulls out the dominant color not shown
 

Weird.

Skipping ahead to the end

This is the pattern: we'll ask a burning question, we'll form this question as a PowerShell pipeline, and we'll see the results.

Question: can we see the images in order of "redness"?

Pipeline:

$a | sort red | % { $_.filename }

Results:

Least red:

 

Most red:

Summary: okay, that makes sense. We used a naive algorithm that simply counted the red value, meaning that a pure black image or a pure blue image would have the "least redness" and a pure white image would have as much "redness" as a pure red image. Hmm, we can fix this. Onwards!

Question: Okay, so we're looking for redness. Let's call this proportional redness. Hmm, here we go:

Pipeline:

$relativelyRed = $a | select filename, @{Name="redness"; expression={$_.red / ($_.red+$_.green+$_.blue) }}
$relativelyRed | sort redness | % { $_.filename }

Results:

Least red:

Most red:

Summary: now that's more like it. Our earlier naive results were instructive, but this is more what I was looking for.

Question: okay, so let's stop messing with redness. Instead, let's find out what images have the most variance between the colors. We're less interested in the white-gray-gray-gray-black spectrum, and are looking for more colorful images. Let's do this:

Pipeline:

$variance = $a | select filename, @{Name="Variance"; Expression={$avg = ($_.red+$_.green+$_.blue)/3; $var = [math]::Abs($_.red-$avg) + [math]::abs($_.green-$avg)+ [math]::abs($_.blue-$avg); $var} }
make-html -fullfilenames ($variance | sort variance | % { $_.filename }) -resultingFilename "variance"

Results:

Most balanced:

Most variance:

Summary: most interesting, besides a grouping of the "grayish" and "black and white" images all together, is the smattering of images that have color, but are so perfectly balanced they're nestled right in there with the pure black-and-white images. Neat.

Final bits

This post is already too long. There's not too much else to say, besides a) stuff is awesome, and b) with the aid of either PowerShell functions or .NET library calls, you can do some complex things. If you only remember one thing from this post, try and pick up the impression I'm trying to leave. This is how I see PowerShell: it's an experimental playground where I morph a thought, an idea, slowly into something workable, and in each step along the way, I'm getting feedback and refining, and in the end, I've satisfied my curiousity. Maybe it's something as useless as basic image analysis using System.Drawing.

Incidentally, if you want to see how the professionals do this kind of thing, check out Multicolr - an color search engine indexing 10 million Flickr pictures, which makes the stuff I did above kind of pitiful looking :) When I checked last, the Multicolr site was slow, otherwise it's neat; check it out.

Friday, May 16, 2008 7:19:58 AM UTC 

I am sometimes asked what PowerShell is. While it's easy enough to give a technical overview, today I'll focus on everything else.

PowerShell is not a fly-by-night scripting tool

It isn't JScript, and won't disappear anytime soon. For the three JScript enthusiasts remaining in the world, I apologize for the cheap shot. Wait, no I don't. It's JScript—no one's using it, no one will be offended.

PowerShell has had a rapid rise since it reached beta status in late 2006, and if you're wondering if this positive trend will continue, let me lay it out for you eloquently:

WHY YES IT WILL, AS A MATTER OF FACT

You may have already heard that Exchange 2007 uses PowerShell for its administration console. Which is great, but I personally couldn't care less—I've got bigger fish to fry.

SQL 2008: PowerShell support. Still don't care, but we're getting warmer.

IIS 7: PowerShell provider. Ok, I like this. Still in beta though, we'll give it some time before any "visible enthusiasm."

administrative duties for Microsoft products going forward - pretend the link to the left is from an authoritative source. Because had I better searching skills, it would be. The point is: major Microsoft server products will begin to support PowerShell in the same way as Exchange 2007 (which again, we couldn't care less about).

If all that hasn't convinced you, maybe the visual aid below will:

 

In Windows Server 2008, PowerShell is an available Feature! We're beyond installers; PowerShell is now built into the operating system! This is even true of the older Operating Systems—if you don't believe me, try uninstalling PowerShell! Can't find it? That's because (in the older OSes anyway) it's installed as an update to Windows, as a standalone hotfix. Hotfix, hotness! If I was rapping I could make "hotfix" and "hotness" rhyme. I could  pull it off.

PowerShell is not going to replace the command shell (cmd.exe)

I briefly thought this would be the case, then Server 2008 Core dashed my hopes. While I admire the engineering-won-over-marketing victory that allowed Server Core to ship with almost absolutely nothing, including all of the .NET framework, I wish we could have simply declared 2008 as "year of the PowerShell" and Server 2008 as "the painful version for the PowerShell laggards." That would have been awesome.

But, as it turns out, Server Core did indeed ship without PowerShell, and as such, we still have cmd.exe available to us, awful parsing rules and all (e.g. try the following: echo "greater than sign is >" - I assure you that will not produce the desired effect). Anyway.

You are not wasting your time learning PowerShell

First, I'll point out that PowerShell is actually quite easy to pick up, and that you can do so with a single PowerShell book. Furthermore, I'll point out that the currently-in-pre-alpha-V2 version of PowerShell is additive and does not contain any breaking changes. Like C#, PowerShell will grow, only. And grow, awesome!

PowerShell doesn't have to be slow

I've read that the PowerShell process will be slow loading the first few times. Which is terrible, because many of us won't make it past those first few times, especially those of us who are cynical of the whole thing to begin with. I assure you, it gets faster.

PowerShell is useful

I'm not telling you to pick up PowerShell to satisfy your yearly programming language learning quota. I'm also not telling you PowerShell will change the way you code in Java or COBOL or SNOBOL or FORTRAN or RPG or whatever. I will instead say that I have saved myself time (and more importantly, aggravation) with PowerShell, whether by spelunking the SharePoint object model, completely automating tasks, or running scripts as post-build actions on my Visual Studio projects…

PowerShell saves me time. And if it doesn't save me time, then it's saving my sanity. And if it isn't doing either of those two things, then it's probably pulling in some mixture of .NET framework classes, using enhanced filesystem/XML/registry support, AD stuff, COM objects, and yes, even running other console applications—it's pulling any or all of these things into one environment, in a way you won't see anywhere else. PowerShell is the de-facto scripting environment for Windows—in fact I used to call it "Perl for Windows, except Perl for Windows already exists, and actually competes directly with PowerShell." Yes, I can't help myself when describing PowerShell and say the quote in full, all the way to the end of the long rambling sentence (like this one (it doesn't seem to ever end)). Well, let me set the record straight as to PowerShell and Perl:

PowerShell is better than Perl

There, I said it.

Well, for Windows, anyway.

Wednesday, April 23, 2008 10:56:08 AM UTC 

Update: welcome to the new-and-improved recap! This recap, unlike the last, weighs in at less than a million billion byes of text and code highlighting and will now load in your feed reader.

This is a recap of my entries into the 2008 Winter Scripting games. From, like, the winter. I know, it's April, we're already in summertime mode here in Texas, and it is quite clear that it is no longer 'the winter'. Let's move on, shall we?

I'm going to work through each of the following scripts illustrated in the table below (the counts for "Lines", "Words", and "Characters" mean what you would imagine they mean):

Compare the above table to MOW, who, for example, had a 1 line, 8 word, 58 character solution for #6. My solutions are (relatively) HUGE. So my recaps, which contain the full source, will necessarily be HUGE as well.

For each problem, I'm going to quickly sum up the interesting (at least TO ME) bits of each problem, then I'm going to post the full source.

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:47:17 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 10: Blackjack!

Here we are asked to simulate a text-based Blackjack game. Mine works, I'm certainly not proud of the graphics, not proud of the colors, not even necessarily proud of the structure of my solution.

Please do not look at any of the following code and think "that's how I'm supposed to use objects in PowerShell!" Please don't. It's NOT ideal, I made several design mistakes; I don't even think most of my objects are necessary. Or, thinking from the other side—maybe I didn't go far enough.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:46:56 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 9: You're Twisting My Words

Simple: reverse letters in each word, WITHOUT reversing the order of the words. It's a simple case of breaking down the huge string into tokens, then reversing the contents of the tokens, then re-combining the tokens. Actually it's even simpler than I described. Moving on.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:46:35 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 8: Making Beautiful Music

In this problem, we are asked to generate a random playlist for our CDs. I'll just say: I cheated. If you look carefully, you'll notice I sorted the songs from longest to shortest, making the "song packing" algorithm more likely to succeed. But it's not perfect and can fail.

So, in summary, I solved the problem like everyone else, but probably not well enough.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:46:13 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 7: Play Ball!

Here we are asked to randomly schedule games for six teams in a round robin tournament. Apparently I used Knuth's classic list order randomization algorithm, which I remember attempting to implement, badly, and failing!,  in high school. Yay for progress.

Otherwise the solution is straightforward: two nested loops, generate matches in a "combinatorial" way, then randomize the order. Boom, done.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:45:50 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 6: Prime Time

Event: calculate primes up to 200. Solution: boring brute-force method.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:45:30 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 5: You Call That a Password?

 

First off, before I describe this event, I would like to protest most all of these rules as bad IT practice. Unnecessary password rules just make your users angrier. ANYWAY.

In this challenge we are told to score passwords based on a set of specific rules. What's awesome about my solution is I use the hotness that is metaprogramming. Okay, so, it's programming hotness. I don't claim that programming hotness in any way translates to actual, real-life hotness, I could be wrong.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:44:58 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 4: Image is Everything

 

In this challenge, we are asked to pretty-print a calendar from text.

My solution is ugly. I don't make excuses, it's ugly. I don't know if I could have cleaned it up any without adding a lot of complexity/structure, so, this is honestly as pretty as I could make it in PowerShell.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:44:34 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 3: Instant (Runoff) Winner

 

This is not as straightforward as it looks. We are presented with a list of votes, each person voting for their preferred candidates, in order of preference. For each vote, you need to first count them as if there are 4 candidates, then count as if there are 3, then 2, until a candidate gets the majority vote. I'm actually proud of my solution. Some of the awesomeness is embodied in the following representative line:

$runoffCandidates = $results | Sort Percent -desc | select -first ($results.Count - 1) | % { $_.Name }

This is the magic that removes the candidate who just got booted off the island by reconstructing the list of the surviving candidates.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:44:08 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 2: Skating on Thin Ice

Here we do some very basic number crunching: read in scores, cut off the small and the large score, calculate the average. Then sort the results and print the top three.

Actually, in describing this I realize that this is a mostly boring problem. Check out how I use PowerShell's support of collections to chop off parts of them, sort, etc. Otherwise, there's not much here.

I will also point out that many of my solutions are excessively Object Happy. I.e., whenever possible, I construct an object with properties. Most of these times (with the exception of the blackjack game, #10, and the election runoff, #3 immediately below), objects were unnecessary. So, let this be your warning.

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Wednesday, April 23, 2008 3:43:42 AM UTC 

If you aren't already in the know, these ten problems are from the 2008 Winter Scripting games.

For each problem, I'm going to quickly sum up the interesting (interesting TO ME) bits of each problem, then I'm going to post the full source.

Event 1: Could I Get Your Phone Number?

This builds a hash table that translates phone numbers to dictionary words (if they exist). That's all it does.

To solve this problem, I reduced it to its very essence: a hash table. The keys of the hash table are the phone numbers. The values are dictionary word(s) that correspond to a phone number.

Representative line:

$ns[$phoneNumber]

Now wasn't that easy?

Source

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Tuesday, February 26, 2008 2:00:54 AM UTC 

One feature of which I was reminded recently by my non-blogging friend John is PowerShell's support for exotic variable names. Let's try this out by example:

Yes, that was a single-quote mark, and yes, I did just name a variable using three exclamation points. That just happened!

I'm not telling you to go out there and start naming your variables with exclamation points because you're feeling chipper that day, even though, by the way, that's an excellent way to express yourself. I'm not telling you to do that.

What I am trying to say is that this "exotic variable name" feature is useful dealing with PowerShell's support for things like CSV import. Importing data using PowerShell's Import-CSV cmdlet automatically assigns the first "header row" as its property names. Sure, you're saying: who cares. Right. We name our header "CustomersFirstName", it comes out the other end $line.CustomersFirstName. YAWN!

 

While the collective "we" name our table headers in such a way that they won't break anything, just because we've been burned so many times, this defensive way of thinking does not always extend to whoever is supplying you with your CSV files. Or, let's just admit it, they're actually exports from someone's Excel spreadsheet. We'll definitely talk about working with Excel some other time; for now, it's back to the variable naming magic. Whereas before you'd say

foreach ($line in $importedCsvFile) {

    Write-Host $line.CustomersFirstName
}

now you can do::

foreach ($line in $importedCsvFile) {

    Write-Host $line.{Customer's First Name} #magic!
}

It's a subtle difference, but when you don't have control over the other guy's data, this sort of subtle issue can be a dealbreaker. Or, thinking positively, a deal-maker! Or, thinking positively but subversively, positively, definitely a dealbreaker! Or thinking non-positively, dealmaker buzzkill! There we go; I'm glad we all made it to the tail end of this paragraph. It's been a rough ride, but we persevered. Congratulations everyone on pushing through! You're all champions to me.

Metaprogramming

I'll mention in passing that, in addition to the ${exotic variable name} and $someVariable.{some exotic property name!!} syntax, we're also granted direct access to the object's methods, properties and other metadata via the $someVariable.PSObject property, which is bolted to every object. Check it out if you're interested; I've found real uses for it a few times. Moving along.

String syntax

We're also granted an odd syntax that allows strings to be used to look up a property. For example:

$worthlessExample = "This is a worthless example, sorry."

$worthessExample."Length"

As you might imagine, $worthlessExample."Length" gets the Length property of the string. That's all well and good, but let's spice it up, shall we?

$dynamicProperty = "Length"

$worthlessExample."$dynamicProperty"

Boom! There's some inline evaluation magic.

The PowerShell book

I would be remiss if I failed to mention that I learned all this from a single source: Bruce Payette's PowerShell book. This book has literally everything you need to know about working with PowerShell, which is quite a statement, especially when you compare it to my library of SharePoint books. So many SharePoint books, and yet so many holes in my SharePoint knowledge! I'm not complaining about SharePoint books so much as I'm trying to express how rare this situation is: you can master this technology with one book!

Monday, February 18, 2008 2:00:51 AM UTC 

I recently finished tearing through the Winter Scripting Games 2008 challenges. I don't remember having this much fun programming at any time in memory; for whatever reason, working through these small exercises was a blast. The only thing that came close to this much programming fun was working through the first few exercises from The Python Challenge (also implemented in PowerShell) several weekends ago.

Nothing else, at work, at school, at home, over the many years, even comes close.

Is it a bad thing, where making an (awful, I assure you, awful) console BlackJack game was more fun than any other programming experience I can remember? I'm going to go with: yes.

2008 Winter Scripting Game Events: Index

• Introduction: TOTAL DOMINATION!
• Event 1: Could I Get Your Phone Number?
• Event 2: Skating on Thin Ice
• Event 3: Instant (Runoff) Winner
• Event 4: Image is Everything
• Event 5: You Call That a Password?
• Event 6: Prime Time
• Event 7: Play Ball!
• Event 8: Making Beautiful Music
• Event 9: You're Twisting My Words
• Event 10: Blackjack!
• Recap

Monday, October 15, 2007 12:00:13 PM UTC 

I find it difficult to explain why I bother using PowerShell with SharePoint; it isn't easy to find a killer example. What I'm about to present is also not a killer example—I just figure that, after a while, I'll wear you down with enough mediocre examples that you'll be convinced. Or tired enough to give up resisting; either is fine with me.

Today's mediocre example

Introducing your team's new knowledgebase! To help everyone get started, I've already created the site and a wiki library (wiki libraries are not exactly the same as wiki sites; the differences are meaningless in the context of this discussion).

We're going to call this wiki library "Yet another soon-to-be abandoned knowledgebase", in an odd naming coincidence that I'm sure has no basis in reality. Everyone I know has plenty of time, incentive and managerial support to document all critical and common processes in an organized and thorough manner. I'm sure your experience mirrors mine.

In the screenshot below, you'll note we have created our knowledgebase.

 

 

Looking at the list view of this wiki library, you'll also notice that our team members have done their part, quickly and efficiently creating articles. Whoa there—watch your back, Wikipedia!

 

Looking at individual entries (like the one below), you'll notice that each entry begins with its own specially formatted title. We have all been so good as to consistently apply this convention across the entire wiki.

[Because I can't leave this alone, I'll note that no, those aren't real keys. Or if you find out that they are real keys, dude! DUDE! Let me know that I'm sitting on a gold mine here!]

 

Only one thing remains: now that everyone is finished with their articles (and let's face it, once the initial push is done, this knowledgebase is as full as it's going to get), you still need to create the wiki home page! Now how would we go about performing this repetitive task against hundreds, or thousands of regex-matchable wiki pages?

Enter SandmanPowerShell

Did someone say PowerShell? For the skeptics, this is where you make the "when all you have is a hammer, everything looks like a nail" joke. Go ahead; I can take it.

Comments before the code

The following code generates an unordered list of all the wiki articles, with the final goal being me pasting this unordered list directly into the home page. This admittedly is not the most exciting or shocking of PowerShell's uses.

The point is: I wasn't typing this data manually; and I wasn't attempting to use some other semi-automated, semi-manual method of get this data into the wiki home page; and I wasn't writing XSL transforms to display the data dynamically. Actually, XSL sounds like a pretty good idea, now that I think about it.

PowerShell code

Without further ado:

 

The Grand Finale: did it work?

You be the judge.

Emitted HTML (I pasted this directly from the clipboard):

<ul>
    <li>[[How To Use This Wiki Library]]</li>
     <li>[[Home]]</li>
     <li>[[Previous knowledgebases|Links to all of our previous knowledgebases]]</li>
     <li>[[Possibly incriminating information|Software activation keys]]</li>
     <li>[[Copy-Pasted Email 1|Re: Re: Fwd: Fwd: contact info]]</li>
     <li>[[Copy-Pasted Email 2|Re: Reboot the server?]]</li>
</ul>

 

Emitted HTML put to use:

Ok, well, it worked. Even for larger than average knowledgebases like my example above, five items aren't enough to justify all this. But, what about for 20 pages?

30?

50?

100?

1000?

etc?

ET CETERA?

At what point does PowerShell become too awesome an option to deny?

I'll wear you down, eventually.

Take-aways

NUMBER ONE: PowerShell is awesome.

NUMBER TWO: Even if you have no intention of using PowerShell, at least realize that through some informally-developed solution, you can work with SharePoint Wiki data programmatically. Whether it's a console application you run on the server, or a Feature you staple to the wiki's Actions menu, or some other Rube Goldberg-esque device, the point is: you can work with SharePoint wikis programmatically.

A short word about wikis (and hackiness)

The example above does not mirror reality; my real-life PowerShell script was much uglier, much hackier, and took almost no time to develop. In this example, I also skipped an intermediate Excel step (yes, Excel). The hackiness of the Excel step was such that it was unfit for publishing.

As for wikis: I'm aware that the wiki home page shouldn't necessarily point to every page in the wiki. I will say this definitively: there is no harm in linking to as much as possible off of the main page, in whatever form, whether it's an unordered list or otherwise.

Friday, September 14, 2007 6:08:22 AM UTC 

On the most recent DotNetRocks TV episode, Carl and Scott demonstrate PowerShell: both its concepts and features, and a lot of how generally sweet it is. Unfortunately Scott ran out of time before he got to the good parts (he was just getting to object piping). Clearly there's room for more PowerShell.

One positive thing about the show is that after watching it, I was inspired to write this post.

Outline

I'm going to cover as many random useful things I've found inside PowerShell as humanly possible. Many of these things will be stupid simple; the common pattern you should notice is that, one way or the other, these scripts save me time, sanity, or time and sanity.

Let's begin!

ONE: Calling into DOS

One of the first things I tried to do in PowerShell was a recursive directory listing. In the CMD shell (the shell formerly known as DOS), the appropriate way to do this is dir /s.

It was quite a shock to find out PowerShell's dir command doesn't support a "/s" switch. Had I known back then what I know now, I would have felt better—you can run any DOS command (including those defined in the command shell itself) from PowerShell simply by prefixing the command with "cmd /c".

I'm not saying that you should be typing "cmd /c dir /s" to get a directory listing—you should be learning PowerShell's superior dir command. But I think it's also important to note that you can get the job done today, immediately, inside the PowerShell prompt, without learning any PowerShell. Just prefix your DOS command with "cmd /c".

I'll enumerate some useful things I've found, even after learning PowerShell:

cmd /c start .

(this opens the current directory in an Explorer window. The "start" command also opens documents and URLs)

cmd /c pause

(this runs the DOS pause command. This is a better solution than a "echo 'Press any key to continue…'; read-host;")

TWO: I'm too lazy to cut and paste—PowerShell is my laziness enabler

I can't say enough good things about the Set-Clipboard cmdlet, and how useful I find it. Here are some of the different things I've run:

(dir)[-1].Fullname | set-clipboard

(dir)[-2].Fullname | set-clipboard

(The above lines are just one act in a five-act play: "How do I copy the full path and name of this file?" 1. Run dir on the entire directory. 2. Eyeball the file you want. If it's close to the bottom, you can use PowerShell's "negative index" functionality to easily count backwards. 3. Get the FullName property of the file, 4. Run the command to copy this to the clipboard, 5. Paste full path and filename elsewhere. I'll note that this hasn't necessarily saved me time.)

(history)[-1] | foreach { $_.CommandLine } | out-string | set-clipboard

(First let me explain what this line does. This pulls the last command from PowerShell's command history, and copies it to the clipboard. in theory, I could have used some other way to get this command into the clipboard—but this is so much easier.)

history | foreach { $_.CommandLine } | out-string | set-clipboard

(This takes the entire history and pastes it to the clipboard. I intend to paste the PowerShell history directly into my Notepad equivalent for pruning into a real script.)

([Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")).FullName | set-clipboard

(This gets the 4-part .NET assembly name, which you'd otherwise have to manually transcribe from your Explorer window's representation of the GAC.  I have used this with SharePoint several times already.)

$a = (sn -T MySharePointWorkflow.dll)
$a[-1].Split(" ")[-1] | set-clipboard

(Assuming the above lines are errorr-free, this gets the PublicKeyToken from "MySharePointWorkflow.dll". If I'm wrong, well, let's all pretend I'm right anyway. Something looking vaguely like this has worked for me once or twice. You should be able to guess by now, but in case you haven't: yes, it's for SharePoint development.)

THREE: Automating SharePoint deployment and administration tasks

stsadm | find """feature"""

First, I'll note that the "stsadm" command is SharePoint's command-line administration tool. I'm running the "find" command to parse the text output and display any lines containing the word "feature". In other words: I'm doing a poor man's grep.

What you may notice above is the odd arrangement of double-quote marks. This gotcha is a side effect of PowerShell's pseudo-hands off approach to interpreting DOS commands. The triple-double quotes above (""") look awkward, but will evaluate down to a single set of quotes once the command is interpreted by DOS. The specifics aren't important. What's important is that once you understand how to escape characters properly like I've done above, you can easily run DOS commands from inside PowerShell. DO IT NATIVE!

SharePoint deployment scripts - I can't tell you how many times I've seen 7-line C# console utilities that are built to supplement missing SharePoint administration functionality. While a small C# console application gets the job done, PowerShell is built for this sort of one-time use console utility. If I could get away with it (and I might yet), I would love love love to abstract away all the SharePoint deployment/development/general ugliness that makes me angry, and replace it all with a post-deployment task/small script/everything script.

Another example I've seen recently: we needed to wire up a custom 404 handler and a custom 404 redirect page. "Deployment" of this solution required both updating a property on a SPWebApplication object and copying files in a very specific manner. I'm proud to say I was able to abstract away this ugliness with PowerShell!

FOUR: REPL in .NET!

EXAMPLE: I need to figure out how the System.Uri class works. Aside: if you're working on web apps and you're unaware of the Uri class, you will want to take a look at it. Moving on.

So in doing quick research on the Uri class (or any class really), there are a few options: MSDN (ha), Google (ok), in-process debug-time experimentation with the Visual Studio Immediate Window (ok), Lutz' Reflector (often excellent), or PowerShell (yessssss). I don't know if you've guessed by now, but we're going to inspect and try things out on the Uri class with PowerShell.

$u = [Uri]"http://test.site.com/testing?query=1&query2=a&query3isempty=&query4=something"

$u
$u | gm
$newUri = new-object "System.Uri" -arg $u, ([Uri]"http://some.other.site/letstry/relative/").PathAndQuery

$newUri

What's awesome about a read-eval-print loop (REPL) is that it reduces experimentation time from minutes (or hours!) to seconds. Actually, anyone who has used any scripting language will be able to tell you why REPL is awesome; feel free to read THEM instead of ME. I hear LISP has been around for a while; maybe you could ask your grandparents.

Thursday, September 13, 2007 5:23:38 AM UTC 

The title above subtly implies I'm some kind of professional SlickRun enthusiast. Which I am. I'm running SlickRun on three physical computers, two functioning virtual machines, and until recently, one virtual machine I lost in an accident (RIP SharePoint VM, E 2007 Eternal). SlickRun is one of the first things I install on any new machine I encounter.

As I find SlickRun's default settings intolerable, one of the first things I do post-install is click-click-click uncheck uncheck delete key delete key tap tap tappity. (All this creative onomatopoeia is the sound of me at work configuring SlickRun). Since I've installed SlickRun fresh 6+ times recently, I've pruned my configuring down to the Best of the Best—both the literal best of the best efficient configuration, and in a metaphorical sense, the Best of the Best karate movie. That was an awesome karate movie.

Oops! I forgot: Introduction to SlickRun

SlickRun, for those of you who are not aware, is a productivity-enhancing unobtrusive program launcher that enables you to:

• Throw away that cluttered Start Menu!*
• Throw away that cluttered Bookmarks/Favorites folder in your browser!**
• Throw away that other productivity tool you use for Google Searches!***
• Throw away all your shortcuts on the Desktop!****

* metaphorically
** metaphorically
*** metaphorically
**** metaphorically

The SlickRun manual, in ten (10) words or less

• WINDOWS KEY + Q  opens SlickRun
• At SlickRun prompt:
  • HELP
  • SETUP
• ESC closes

Essential Post-Install Configuration Steps

If nothing else, at the least make these two changes:

ESSENTIAL #1: Make SlickRun invisible

I actually didn't like SlickRun when I first installed it. Why? Because SlickRun's out-of-the-box experience is an annoying one—by default, SlickRun adds itself to the system tray and throws an obnoxious blue bar in front of all other windows.

Thankfully we can hide SlickRun, hide the system tray icon, and restore balance to the world. I don't want to overstate the importance of this, but you may just save someone's life—and that life may be your very own! Proper SlickRun configurations save lives! But I don't want to overstate this.

Anyway: In the Setup screen, configure the Options tab:

TIME ELAPSED: +10 SECONDS.

ESSENTIAL #2: Get rid of the awful preset MagicWords

SlickRun comes bundled with one (1) useful MagicWord: the Google one. Everything else is useless and needs to go.

You can use the mouse and click your way through the deletions, but I've found the magic keystrokes to be:

• DOWNARROW to highlight the next MagicWord
• ALT-D to perform a Delete operation
• Y to confirm deletion.

Repeat rapidfire DOWNARROW, ALT-D, Y until you've deleted everything. Except the "google" MagicWord; that one's golden.

TIME ELAPSED: +20 SECONDS.

Useful Configurations

I don't want to present the following as MUST HAVES; instead, I'll just say that I have found the following to be personally useful, and I believe you will as well.

USEFUL #1: Make the font massive

This picture expresses everything I want to say on the subject:

Citation: props to Ben for this tip. Detailed instructions are posted immediately below:

TIME ELAPSED: +10 SECONDS.

USEFUL #2: Google search at the prompt

POP QUIZ, HOTSHOT: how few keystrokes do you need to run a search?

With my SlickRun "g" MagicWord, three (3).

(I am aware Vista has improved the search experience such that it is reduced to literally one keystroke. That's totally awesome, and I'm sure we'll all be enjoying this "one-keystroke search" feature when Vista R2 SP1 arrives sometime in 2009).

I will also note here that SlickRun has a form of autocomplete that is similar to typing an address into your browser—if you've typed your phrase before, SlickRun will "guess ahead" at what you intend. What I mean to say is that SlickRun finds more ways to make itself useful!

TIME ELAPSED: +20 SECONDS.

USEFUL #3: Browser-targeted favorites

Let's assume your default browser is Firefox and not Internet Explorer. Let's just say. How do you open up an IE-only site in the browser?

• The Firefox plugin IETab works wonders, but IETab does consistently crash with some of our more "creative" legacy intranet sites. IETab, for those of you who don't know, allows you to visit specific sites within an IE browser hosted inside of the Firefox window. So it looks like Firefox, but it renders (and behaves) like IE. It's awesome for 90% of the site that require IE.
• Set up a SlickRun MagicWord for this browser-targeted favorite in 10 seconds. SET IT AND FORGET IT!

We'll focus on SlickRun today. Let's try doing this by practical example (this is something I have set up today):

EXAMPLE: You need to connect to your Virtual Server administration website. This site runs a heavy-duty ActiveX control and thus should only be opened inside of IE. Your default browser is (by accident I'm sure) set to Firefox. What is a quick, painless way to access the Virtual Server site?

Enter SlickRun:

TIME ELAPSED: +30 SECONDS per URL

 USEFUL #4: Program launching

It should be obvious to note that we use SlickRun to launch programs.  Duh. What is not obvious is that you probably shouldn't set up all your programs in a "SlickRun big bang go-live"; instead, add new programs to the SlickRun launcher only when you realize, for example, "I use Paint.NET a lot." If you add too many programs at once, you won't remember your SlickRun aliases for them. As Nacho Libre says, "Take it easy!"

TIME ELAPSED: +30 SECONDS per program

USEFUL #5: Document shortcuts

I read eBooks, and I'm not ashamed. Go ahead, poke fun.

I'm also unashamed to say that I haven't memorized all 1100+ pages of this SharePoint administration book. Yet. In the interim, I use the very handy eBook provided on CD (aside: did you know Microsoft Press provides the full PDF eBook of this SharePoint administration book at no additional charge? Use it!).

I use this eBook often—and what is most telling about how much I rely on SlickRun is the fact that I have no idea where on my hard drive I copied the PDF—only SlickRun knows. I "set it and forget it" in the purest sense.

TIME ELAPSED: +30 SECONDS per document shortcut

Extreme SlickRun

I've been experimenting with running small PowerShell scripts whose sole purpose is to  copy something to the clipboard. To accomplish this, I'm using the fully-functional Set-Clipboard cmdlet (thanks Keith Hill) that is bundled in the PowerShell Community Extensions.

These "clipboard scriptlets," while useful when I type them into the PowerShell  interpreter, are so much more useful because I can launch them directly from SlickRun. One option of running scripts is the traditional way:  create a script file, and set up a command to run the script file (e.g. "powershell.exe C:\a\get-something.ps1"). A new option we're presented with is to use powershell.exe's "-command" parameter—sometimes I embed all of the PowerShell code inside the -command parameter, meaning these little clipboard scriptlets reside entirely inside of SlickRun.

One example of the simple little things I did was create a random GUID inside of PowerShell and copy the new GUID onto the clipboard. Because (for whatever reason) the GUID tool in Visual Studio was not available, this was my best option. And yes, SharePoint is at times GUID-hungry, (SharePoint is the hungry hungry hippo of GUIDs, and yes, you can quote me on that) so I know I've used this a dozen times at least.

powershell.exe -command { "$([Guid]::NewGuid().ToString()| set-clipboard)" }

Anyway, while this [Guid]::NewGuid().ToString()| set-clipboard is easy enough to run in an open PowerShell prompt, it literally took 15 seconds to set this alias up in SlickRun. SET IT AND FORGET IT!

TIME ELAPSED: TOO EXTREME TO ACCURATELY CALCULATE (OUTSIDE OF TOLERANCES)

Download SlickRun

Download SlickRun!

Check out Bayden Systems, maker of SlickRun!

Wednesday, August 29, 2007 4:13:26 AM UTC 

The magic representative line

The single line (#17) in the image above calls stsadm.exe (a console command) and passes several arguments to it: 1) a bunch of SharePoint stsadm junk/raw text, and 2) the GUID of the feature I was trying to remove.

What is certainly most uninteresting is the SharePoint details; this was development-VM administrivia. What is certainly interesting is that I have run the PowerShell interpreter where needed to grab the Feature's GUID, but sent the rest of the text directly to the console application without surrounding anything in quotes. Try and point out, on line #17, where DOS ends and PowerShell begins.

Thus, magic!

Aside from all the interpreter nuttiness, this was super useful and saved me heaps of time*, so there was some practicality involved as well.

*heaps of time => probably 20 seconds or less

Thursday, July 19, 2007 6:50:39 AM UTC 

I recently read Simon's post about using Reflector with SharePoint. Inspired, I set about to use Reflector as a sort of "MSDN 2" to help me work through holes in MSDN's (and to be fair, the rest of the world's) documentation. Unfortunately for me, I had to track down DLLs and ASMX files from different subfolders of the 12 hive and even from the GAC.

I'm writing this post to help me (and you) find the SharePoint 2007 DLLs and EXE's and everything else we might need to reverse eng…um, let me rephrase that, "embrace a lifestyle of self-service SharePoint documentation." That last bit sounds nicer, yes.

For various non-technical reasons, I can't just post the code directly here—instead, we must each go to our MOSS (or WSS) farms and copy out the DLLs. Thus far I've found three directory trees SharePoint is storing its code:

Name	Directory	SKUs	Fun Facts!
The 12 Hive	C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\	WSS, MOSS	What's sad is that I typed the full directory from memory—including the awkward capitalization.
The "Office Server" directory	C:\Program Files\Microsoft Office Servers\ (can be customized)	MOSS	Post your fun fact in the comments, otherwise I'll have to stick with "Fun fact: Excel Services .dllis hosted here!" See, you can do better.
The GAC	%WINDIR%\Assembly\	WSS, MOSS	Windows Explorer hides files in the GAC from you (it's a special folder), so you'll have to work a little to get at them.

The added value I'm here to present today is, well, I automated all of the copying—I wrote a PowerShell script to grab all the files. It's way overkill (especially the Vista-style nuttiness when I spend half of the script calculating the estimated time the other half will take to complete), but hey, why not?
Here we are:

Find-SharePointDllsVistaEdition.ps1 (2.06 KB)

#Freely distribute and use. Absolutely no warranty is implied.
$dirTwelveHive = "C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\"
$dirMossServers = "C:\Program Files\Microsoft Office Servers"

$subDirectory = read-host "Full pathname to write files?"
mkdir $subDirectory -ErrorAction SilentlyContinue | out-null
mkdir ([System.IO.Path]::Combine($subDirectory, "12-hive")) -ErrorAction SilentlyContinue  | out-null
mkdir ([System.IO.Path]::Combine($subDirectory, "MOSS")) -ErrorAction SilentlyContinue  | out-null
mkdir ([System.IO.Path]::Combine($subDirectory, "GAC")) -ErrorAction SilentlyContinue  | out-null

echo "Calculating time required to perform this operation…"
$excludedFileExtensions = dir -path "$([System.IO.Path]::Combine($dirTwelveHive, ""template\images""))
" | foreach { $_.Extension.ToUpper() } | group | foreach { $_.Name }
$excludedFileExtensions += ("MDF","LDF")
$exclusionCriteria = $excludedFileExtensions | foreach { "*$_"}

$allFiles = dir -path $dirTwelveHive -recurse -exclude $exclusionCriteria -ErrorAction SilentlyContinue
$allFiles += dir -path $dirMossServers -recurse -exclude $exclusionCriteria -ErrorAction SilentlyContinue
$allFiles += dir -path ([System.IO.Path]::Combine($subDirectory, "GAC")) -recurse -include "*SharePoint*.dll" -ErrorAction SilentlyContinue

$allFiles | Select Extension | group Extension | select Name, Count | sort Count -desc | select -first 10

$sum = 0
$allFiles | foreach {
    $sum += (dir $_).Length
}

echo "`n`n Total size in MB: $($sum/1MB)"

cp -recurse -force -path $dirTwelveHive -destination ([System.IO.Path]::Combine($subDirectory, "12-hive")) -exclude $exclusionCriteria
cp -recurse -force -path $dirMossServers -destination ([System.IO.Path]::Combine($subDirectory, "MOSS")) -exclude $exclusionCriteria

#does not preserve directory structure
$filesFromGac = dir -recurse -path "$([System.IO.Path]::Combine($env:WINDIR,""assembly""))" -include "*SharePoint*.dll"
$filesFromGac | foreach { cp -force -path $_ -destination ([System.IO.Path]::Combine($subDirectory, "GAC")) }

I'd better hit the "Post" button before my laptop battery runs out.

Thursday, July 19, 2007 4:42:28 AM UTC 

In my other, somewhat-less-happy post about CAML, I shared my frustration of working with the MSDN (and other) CAML documentation. What I failed to do in that other post was share the results, i.e. tell you how I eventually got the job done.

This will not bring you all the way from start to finish, but should help you hurdle some of the less obvious…let's call them 'hurdles'. I'm sticking with the hurdling metaphor—there's no point in switching metaphors midsentence, even if the word 'hurdle' is starting to lose all meaning.

Step #1: Go visit the U2U Community Tools page and download both of their excellent CAML query tools. If you stop reading now and click on the U2U links, it's probably for the best.

The two tools are:

• "CAML Builder" - this Windows app that will actually use SharePoint's web services to pull list metadata, meaning that you can actually build your CAML query almost as easily as every other query tool on the planet. The one gotcha is that (in this case) it mingles the three XML chunks together (see code below for an explanation of what I mean by "chunks").
• "DevFeaturesPackage" - yes, that's what it's called. This is a v3-only Feature that attaches itself all lists' Actions menu in your browser. And, it works like every other query tool on the planet.
  

My "works like every other query tool on the planet" quips above might sound sarcastic, but trust me: if you've spent any amount of time working with this stuff, finding things familiar and useful is a great relief.

Step #2: Find your lists.asmx web service URL and add it to your project as a web reference. In my case, and for probably every other case in the entire world, it will be http://servername/site/site/site/_vti_bin/lists.asmx - unless you've done something crazy, like install SharePoint on a subdirectory of the web application. Note that the URL does include the site heirarchy. This paragraph is a longwinded way of saying "server name, plus site heirarchy, plus_vti_bin, plus lists.asmx".

You can be absolutely certain by running your lists.asmx URL through the U2U CAML Builder and verifying you're connecting to the correct site.

Step #3: Build your CAML query. I'm not going to attempt to help with this; instead, I'll refer you to the U2U CAML query tools again.

Step #4: Write code to build up the six (6) parameters this web service takes. I've posted the PowerShell function I used to do this, and annotated where necessary (which is everywhere).

Step #5: You're pretty much through! Now all you have to do is bask in the glory of a job well done! Don't even test! You can't test—you're too busy partying and you got it right the first time! Woo!

Well, you might want to test.

Footnote: My PowerShell code, doctored some—if you plan on copy/pasting this, you'll probably want your code…cleaner. Without further ado:

#—
#Tested only against WSS v2. Presumably also works with v3 sites.
function Get-ListItems ($webService)
{
   
    $listname = "YOUR_LIST_NAME_GOES_HERE"

    #Empty string indicates we use the default view. Otherwise, we'll have to provide the view GUID. I think.
    #Presumably using a custom view is an excellent way to set a filter on your data set.
    $viewname = ""

    #I don't know what the rowlimit means, and though I can GUESS, I don't want to ASSUME.
    $rowlimit = "100"

    #I created an XmlDocument object ONLY so I could create valid XmlElements for the 3 web service arguments.
    #$x is not used directly.
    $x = new-object "System.Xml.XmlDocument"

    #<Query> - presumably you can set attributes or inner XML if necessary.
    $q = $x.CreateElement("Query")

    #<QueryOptions> - presumably you can set attributes or inner XML if necessary.
    $qo = $x.CreateElement("QueryOptions")

    #I'm posting my "ViewFields" element as-is. The goal of my query was to pull the ID of every
    #item visible in the default view. I will assume that your ViewFields element will have more content.
    $v = $x.CreateElement("ViewFields")
    #Don't be thrown by the odd PowerShell string syntax. "" inside a quote is like \" in C#.
    # the .set_InnerXml is just one of PowerShell's ways of representing .NET's property set operations.
    $v.set_InnerXml("<FieldRef Name=""ID"" />")

    #I can't remember why this was necessary anymore, but is probably related to my ignorance of the XmlDocument object.
    #If I'm guessing correctly, SharePoint requires XML submitted as <Query></Query> instead of <Query />. But that's guessing.
    #Either way, feel free to ignore these two lines and see if you can get your lists web service working without them.
    $q.set_InnerXml("")
    $qo.set_InnerXml("")
  

    $result = $webService.GetListItems($listname, $viewname, $q, $v, $rowlimit, $qo)

    #this is awkward PowerShell syntax that could probably be replaced with some proper XPath.
    $trimmedResult = $result.data.row | foreach { $_.ows_ID }
  
    #this line means "return trimmedResult"
    $trimmedResult
}
#—

Wednesday, March 28, 2007 5:31:53 AM UTC 

This little gem may come in handy when you think you may need to do some performance tuning on your PowerShell script. I'll just let my brilliant script do all the talking:

—
function Help-PerformanceTuning
{
      #this function helps me when I feel the urge to
      #optimize for performance
      $max = 4000

      $regexObjects = @()
      for($i=0; $i -lt ($max-1); $i++)
      {
            $regexObjects += [regex](Get-TwelveRandomCharacters)
      }
     
      echo "Congratulations! You've created $max Regex objects! That's totally, ridiculously wasteful! Now get back to work!"     
}
—

It sure cures what's ailing me!

Astute readers may have noted that I call a "Get-TwelveRandomCharacters" function in there. Well, we should all be pleased to note that I further waste resources by instantiating twelve Random objects for each Regex object created! Let's do this:

—
function Get-TwelveRandomCharacters
{
      $chars = @()
      for ($i=0; $i -lt 12; $i++)
      {
            $r = new-object Random
            $chars += [string][char][byte]($r.Next(1,26)+64)
      }

      [string]::Join("", $chars)
}
—

Awesome!

Wednesday, February 28, 2007 3:38:31 AM UTC 

Nuttiness enumerated below:

• UPDATED: Misinformation removed—the PowerShell Book has set me straight. Inside a PowerShell function, all method calls returning a value must be prefixed with [void]. This means mostly .NET method calls and COM object methods. If you don't use the [void] prefix, your $xls.ActiveWorkbook.Save() operation returns a COM object to the calling function, not exactly "expected behavior." The behavior isn't a problem now that I know.
  
• The file redirection operators (in my case, the >> append operator) output in Unicode by default. This is fine, except that I was appending to an ASCII file. Whoops! The (working) code ended up as:
  #not exactly as quick and easy as "string" >> $outfilename, but at least it exists
  "string" | out-file -filepath $outfilename -append -encoding "ASCII"
  
  Footnote: the PowerShell Book also covered this specific scenario.
  
• mp3info is what I'm using to catalog my podcasts (I'm really just looking for the total run time, and maybe encoding bitrate), and it looks like it's going to work most excellently. What's awesome about PowerShell is that it can easily import CSV or XML (both formats may be outputted by mp3info), but I could have also used a .NET library that reads MP3 files. In this case, (i.e., today,) it was slightly easier to run the console application and parse its text output in PowerShell. Tomorrow, I may use an available .NET (or COM) library. The brilliance of PowerShell is, either way is quite easy.
  
  Footnote: the PowerShell Book also covered text file parsing and handling XML documents. Yeah, definitely recommended.

Update: looks like I've got ~372 hours of podcasts in the queue, weighing in at 12.5GB at an (unweighted) average bitrate of 84kbps in 679 files. Yeah. A new Great Podcast Roundup to follow soon.

Thursday, November 23, 2006 3:10:57 AM UTC 

Introduction

I'll try to summarize as succintly as possible:

PowerShell is the first true Windows scripting language. It's not VBScript. I'm sorry that in the past, I've compared PowerShell directly with VBScript as if they were equals; they're totally not. PowerShell can use COM objects like VBScript, but that's about it comparison-wise.

PowerShell is a real language, closer to Perl and Python (and Ruby) than it is to any flavor of VB (excluding VBx—the future, possibly mythical and possibly vaporous, scriptable version).

PowerShell uses the .NET framework with gusto!

PowerShell is also an excellent place to let a quick two-line script rip, or to explore the SharePoint object model in a sort of immediate window (or what the Lisp, Ruby, Smalltalk and Python folk like to call the "interpreter"). It's inline, is the point.

PowerShell pipes objects, not text (though it does pretty well with text too). This "object piping" nonsense is best explained by example (i.e. not here).

UPDATE 2007-08-06: despite my beefed-up introduction above, it's probably wise to go read someone else's introduction.

Step 1: Disable Security

"Secure by default" security measures have proven to be effective, and this next section is in no way disrespectful towards the important strides the industry has made yada yada yada now how do you turn off security so I can run some scripts already? It's easy! Load up the PowerShell prompt and type in:

Set-ExecutionPolicy Unrestricted

This will give you (and potentially, hackers like Boris) the ability to run scripts just like all the viruses used to do with Word macros and HTML application files. To further open security holes, associate .PS1 files with the powershell executable, found in "%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe".

Step 2: Install an IDE

UPDATED 2007-08-06: PowerShell Analyzer is the only PowerShell IDE of which I know. I am not an heavy scripter, so I can't personally justify a money-costing IDE, but you may be interested. As for myself, I'm using a text editor with PowerShell syntax highlighting.  I'm not strongly recommending either one; pick something—anything but notepad!—and go.

Step 3: Wait NaN Months for Documentation

The best online source for information about PowerShell is a tie between a someone calling himself /\/\o\/\/ and the rest of the internet. That's both a tribute to how excellent MOW's information is, and simultaneously an omen warning you how completely lacking everything else is. Also, there aren't any books—

UPDATE 2007-08-06: //\/\O\\/\/ is still the best source of online information, but at least now there's a PowerShell book, including an excellent "just a PDF" eBook I highly recommend buying (available only direct from the publisher). Though the eBook concept may not sound as awesome as you'd think, note that you can navigate with both search and internal hyperlinks—this is why we invented computers in the first place, right? So we can type "CTRL+F, type "[switch] parameter", hit ENTER" and find out what a switch parameter does? Answer: yes, and to answer the question a second time for emphasis: yes, emphatically.

Step 4: Get the PSCX (Community Extensions)

The PowerShell Community Extensions are an excellent collection of useful cmdlets (i.e., "PowerShell libraries" (i.e., "programming stuff" (i.e., "oh forget it—let's just say 'stuff'"))). One of my favorites is the "Set-Clipboard" cmdlet (it outputs directly to the Windows Clipboard)—not that you particularly care about my favorite; discover your own. You'll find at least one indispensable cmdlet among the bunch.

BONUS DOWNLOAD: Try the  world-famous tab expansion engine (read: PowerShell intellisense, at the prompt), developed by none other than /\/\o\/\/, "the better portion of PowerShell content on the internet".

Footnote: Active Directory (ADSI) support is weird

UPDATE 2007-08-06: I've found an excellent summary of PowerShell's ADSI (or Active Directory) support. This page was not the first result on Google when I searched for "PowerShell ADSI", which is nothing less than a crime—a search engine results felony.

The really, really quick summary is that all of the useful methods for working with Active Directory objects are hidden from you. What should be obvious, isn't. What should be discoverable, isn't.

Let's do this by hypothetical example. Not that I've tried this myself! I'm smarter than that. Let's just say that I wanted to find all members of an Active Directory security group (let's just say). Assuming you already have the group object, how would you walk the object hierarchy to pull all members of the group? I'll make it easy and give you three options:

#option 1
$group.member
#option 2
$group.psbase.get("member")
#option 3
$group.psbase.invokeget("member")

If you guessed "oh no it can't be #3," guess what! And if you're not angry enough, let me tell you that the super bonus of this story is that all three options are legal syntax.

Guess how I spent my time today?

UPDATE 2007-08-06: Again, read that excellent PowerShell ADSI article I referenced above. It explains all this and more.