Synchronizing native Android G Suite apps (GMail, Calendar, etc.) with Outlook.com accounts

TLDR – scroll to the bottom for a checklist of what you need to do

The task at hand seemed simple enough. I had a GMail account (ohad188@gmail.com) and an Outlook.com account (also ohad188@gmail.com) and I wanted to sync both to my Android device using the native apps, primarily GMail and Calendar (I usually prefer native apps to third party ones such as the Outlook app).But what should have been a trivial exercise ended up as a complicated, non-user friendly ordeal. I’ll share my story here for the benefit of others (and future self).

Adding the GMail account was, naturally, easy and effortless. This is virtually the first thing everyone does when they get their phone, so no surprise there. However, when I tried adding my Outlook.com account I got “Duplicate account – you’re already using this username for the account ohad188@gmail.com”. It’s pretty clear what’s going on here. The GMail app is keying accounts by username only, and since my Outlook.com account happens to have the same username as my Google account, I’m locked out. I was very surprised that such a mature app would have such a trivial limitation, but what can you do. I also couldn’t find much about it on the web, so I decided I’d just work around the limitation (after reporting it to Google). The workaround was simple – I created an ohad188@outlook.com alias for my account and made it the primary one. This resolved the duplicate account issue, but the story was not over.

While my e-mail seemed to sync fine, I couldn’t see any of my events in the calendar. I looked in every possible setting with no luck. Finally I found an MS Office support article that provided a clue for the issue. It turns out that when setting up Outlook.com accounts, you actually have to select Exchange in the account type selection dialog – not Outlook! And as if that’s not bad enough, if you happen to have that account already configured somewhere else (as an Outlook account, IMAP account, or Outlook app account), you won’t be able to add it properly (you will get redirected to the Microsoft account sign-in page which is not what you want). So I had to delete all existing references to my Outlook.com account (and removed the Outlook app for good measure) before I could get to the actual Exchange account addition dialog. You will know you’re in the right place when you are asked to provide your password in a native Android dialog (as opposed to a redirected Microsoft sign-in page).

But my troubles did not stop there. After multiple attempts, Android kept insisting that my Outlook.com password was incorrect. Of course, I had just used it successfully when I set up the exact same account as an “Outlook” account a few minutes earlier, and presumable my short-term memory was not damaged in the process. Fortunately, I quickly realized (with no help fom Android or Outlook.com, mind you) that the problem was the 2-factor authentication configured for my account. So I issued an app password, provided it in the Exchange account addition dialog, and finally my account was added – mail, calendar, contacts.

So to sum up:

  1. Make sure your Outlook.com primary alias/username is different from your GMail address.
  2. Remove the Outlook app and all references to your Outlook.com account in Android’s account settings.
  3. Add your Outlook.com as an Exchange account and use an app password if 2-factor authenticatino is enabled on your account.

Tab selects text instead of indenting in Visual Studio

I ran into an annoying phenomena today when I tried indenting some multi-line expression I had in my code. I pressed TAB but instead of indenting, it selected some text. This was driving me mad so I started enabling and disabling extensions (binary search to the rescue) until I isolated the culprit – Resharper.

It turns out that Resharper Ultimate 2016.2 introduced a new Structural navigation feature (enabled by default) that makes tab move text selections instead of indenting when pressed mid-line. I also found out that I wasn’t the first on who complained about it.

Anyway, to disable it simply turn off Structural Navigation in Resharper’s Options (Editor -> Editor Behavior).

Adding reddit RSS feeds to Outlook

Creating reddit RSS feeds is pretty straightforward. For example, subscribing to http://www.reddit.com/r/news/.rss will get you all threads in the news subreddit. However, attempting to add such a URL to Outlook’s RSS reader will result in the following error:

Outlook cannot process the RSS content from http://www.reddit.com/r/news/.rss. The link may not point to a valid RSS source.

Fortunately, the fix is simple – just get rid of the right-most slash (the one before .rss): http://www.reddit.com/r/news.rss.

Outlook will now accept the URL and your subscription will work as expected.

ET4W – generate C# ETW classes from JSON event specifications using T4

ET4W

Event Tracing for Windows (ETW) is the best tracing solution for the Windows platform, period. It is unmatched in performance and reliability. More and more tools are built to analyze its events to amazing depth. Microsoft explicitly recommends it for almost any logging purpose.

Unfortunately, writing proper ETW classes is a bit tricky. It used to be much worse (before the advent of automatic manifest generation), but there’s still a lot of code that has to be written manually in a very specific way. Not only is it tedious, it also leaves plenty of room for user error.

  • You have to decorate each event method with EventAttribute (theoretically optional, but in practice you’ll do this for every single method).
  • You then have to call WriteEvent in a very specific manner – the event ID and parameters must match exactly and in order.
  • Tasks and keywords should be specified using nested classes with a very specific structure.
  • You are encouraged to expose the class in a very specific singleton pattern.
  • You must be aware of the exact types supported by ETW (not documented anywhere, so you have to reflect it off ManifestBuilder.GetTypeName).
  • If you want to log types that aren’t supported, it’s your responsibility to invoke the appropriate conversions (typically via manually created wrapper classes).
  • If you want some common parameters to be present in every event, you have to add them manually to each method, and provide them manually in each call.

Enter ET4W.

This T4 Text Templete based NuGet package will solve all these issues for you. All you have to do is write a JSON file specifying the events you want, as well as the custom types you want to support and the common parameters you want present. There’s even a JSON schema for in-editor validation and auto-completion (supported in Visual Studio and other editors).

ET4W will take care of the rest, creating full-fledged EventSource classes according to all the best practices (both System.Diagnostics.Tracing and Microsoft.Diagnostics.Tracing are supported). In addition, wrapper classes are created to support common parameters and custom types. Finally, a few extra validations are thrown into the mix.

Check it out!
https://github.com/ohadschn/ET4W (Apache 2.0)

P.S. one of the nice thing about this approach is that it lends itself well to cross-platform event consistency. By building similar generation scripts for other languages (VB, C++, etc), you could leverage the same event JSON to maintain a “single source of truth” for your events across various platforms (critical for telemetry measurements, for example).

Invoking arbitrary PowerShell commands with retries in PowerShell

Yesterday I blogged about a function I created to execute arbitrary shell (cmd) commands in PowerShell, with retries in face of failures. But what if the command I want to execute is actually another PowerShell script (or function, or cmdlet)? The rules change a bit:

  • We need to use splatting when making the call (otherwise named parameters won’t work, see the references below).
  • In addition to splatting the call itself, we also need to format the argument hash table for printing the command (otherwise it would just print something like Executing: Foo.ps1 System.Collections.Hashtable).
  • $LASTEXITCODE is no longer (necessarily) relevant, we need to work with PowerShell’s $? automatic variable.
  • Parsing the error stream makes less sense, but we need to handle exceptions.

Here’s how it looks:

<#

.SYNOPSIS
Invokes a command with retries.

.DESCRIPTION
The Call-PSCommandWithRetries function invokes PowerShell scripts, functions, or Cmdlets using the provided parameters, with optional retries in configurable intervals upon failures.

.PARAMETER Command 
The command to invoke. Can be any PowerShell script, function, or Cmdlet.

.PARAMETER Arguments
Arguments to pass when invoking the comand (using splatting). 

.PARAMETER RetrySleepSeconds
Time in seconds to sleep between retry attempts in case of command failure.

.PARAMETER MaxAttempts
Maximum number of retry attempts in case of command failure.

.PARAMETER PrintCommand
Determines whether or not to print the full command to the host before execution.

.INPUTS 
None. You cannot pipe objects to Call-PSCommandWithRetries.

.OUTPUTS
The output of the last command execution.

.EXAMPLE
$output = Call-PSCommandWithRetries "dir" @{"Path"='C:\'}

#>
function Call-PSCommandWithRetries
{
 [CmdletBinding()]
 Param( 
    [Parameter(Mandatory=$True)]
    [string]$Command,
    [hashtable]$Arguments,
    [int]$RetrySleepSeconds = 10,
    [int]$MaxAttempts = 10,
    [bool]$PrintCommand = $True
 )

 Process
 {
  $attempt = 0
  while ($true)
  {
   $formattedArgs = $Arguments.Keys.ForEach({"-$($_):$($Arguments.$_)"}) -join ' '
   Write-Host $(if ($PrintCommand) {"Executing: $Command $formattedArgs"} else {"Executing PS command..."}) 
  
   $exceptionThrown = $false
   try
   {
    & $Command @Arguments 2>&1 | tee -Variable output | Write-Host
   }
   catch
   {
    Write-Host "PS command threw exception: $($_.Exception)" -ForegroundColor Yellow
    $exceptionThrown = $true
   }
  
   if ($? -and !($exceptionThrown))
   {
    Write-Host "PS Command executed successfully"
    return $output
   }
  
   Write-Host "PS Command failed" -ForegroundColor Yellow
   if ($attempt -eq $MaxAttempts)
   {
    $ex = new-object System.Management.Automation.CmdletInvocationException "All retry attempts exhausted"
    $category = [System.Management.Automation.ErrorCategory]::LimitsExceeded
    $errRecord = new-object System.Management.Automation.ErrorRecord $ex, "CommandFailed", $category, $Path
    $psCmdlet.WriteError($errRecord)
    return $output
   }
    
   $attempt++;
   Write-Host "Retrying test execution [#$attempt/$MaxAttempts] in $RetrySleepSeconds seconds..."
   Start-Sleep -s $RetrySleepSeconds
  }
 }
}

References:

Invoking arbitrary shell (cmd) commands with retries in PowerShell

EDIT– I initially tried to support execution of PowerShell scripts, functions, and cmdlets in the code below. However, it turns out that they behave too differently so I wrote a separate post on it: https://www.ohadsoft.com/2016/04/invoking-arbitrary-powershell-commands-with-retries-in-powershell/.

It may sound ironic, but executing shell (cmd) commands from PowerShell is not always trivial. There are various possibilities including Start-Process, [Diagnostics.Process]::Start, the call operator (&), and others. Each has its own quirks regarding exit codes, output redirection, argument escaping, and more. For a taste of the issues you might run into, see this blog post by edgylogic.

Having played around with most (all?) of the options, I finally settled down on what I percieve to be the most robust, versatile, and simple way of doing it. The (advanced) function below will allow the following:

  • Print the command before execution.
  • Inspect the exit code and error stream to determine failure.
  • Retry the command in configurable intervals upon failure.
  • Pass multiple parameters.
  • Emit the command’s output to the pipe for further processing.
  • Set the execution status ($?) to $False in case of failre on all retries.
<#

.SYNOPSIS
Calls a shell (cmd) command with retries

.DESCRIPTION
The Call-CommandWithRetries function calls shell (cmd) commands using the provided parameters, with optional retries in configurable intervals upon failures.

.PARAMETER Command 
The command to call.

.PARAMETER Arguments
Arguments to pass when invoking the comand. 

.PARAMETER TrustExitCode
Trust the command's exit code for the purpose of determining whether it was successful or not. 
If this parameter is $False, a non-empty stderr will also be considered a failure.

.PARAMETER RetrySleepSeconds
Time in seconds to sleep between retry attempts in case of command failure.

.PARAMETER MaxAttempts
Maximum number of retry attempts in case of command failure.

.PARAMETER PrintCommand
Determines whether or not to print the full command to the host before execution.

.INPUTS 
None. You cannot pipe objects to Call-CommandWithRetries.

.OUTPUTS
The output of the last command execution.

.EXAMPLE
Use cURL for Windows to download the latest NuGet command-line client
C:\PS> Call-CommandWithRetries "curl.exe" @("--fail", "-O", "https://dist.nuget.org/win-x86-commandline/latest/nuget.exe")

#>
function Call-CommandWithRetries
{
 [CmdletBinding()]
 param( 
     [Parameter(Mandatory=$True)]
     [string]$Command,
     [Array]$Arguments,
     [bool]$TrustExitCode = $True,
     [int]$RetrySleepSeconds = 10,
     [int]$MaxAttempts = 10,
     [bool]$PrintCommand = $True
 )

 Process
 {
  $attempt = 0
  while ($true)
  {   
   Write-Host $(if ($PrintCommand) {"Executing: $Command $Arguments"} else {"Executing command..."}) 
   & $Command $Arguments 2>&1 | tee -Variable output | Write-Host
        
   $stderr = $output | where { $_ -is [System.Management.Automation.ErrorRecord] }
   if ( ($LASTEXITCODE -eq 0) -and ($TrustExitCode -or !($stderr)) )
   {
    Write-Host "Command executed successfully"
    return $output
   }

   Write-Host "Command failed with exit code ($LASTEXITCODE) and stderr: $stderr" -ForegroundColor Yellow
   if ($attempt -eq $MaxAttempts)
   {
    $ex = new-object System.Management.Automation.CmdletInvocationException "All retry attempts exhausted"
    $category = [System.Management.Automation.ErrorCategory]::LimitsExceeded
    $errRecord = new-object System.Management.Automation.ErrorRecord $ex, "CommandFailed", $category, $Command
    $psCmdlet.WriteError($errRecord)
    return $output
   }
            
   $attempt++;
   Write-Host "Retrying test execution [#$attempt/$MaxAttempts] in $RetrySleepSeconds seconds..."
   Start-Sleep -s $RetrySleepSeconds
  }
 }
}

References:

Creating beautiful WordPress header images for free – legally

Personally, I like using the official WordPress.org themes as they always seem to look professional and clean. Moreover, they are very unlikely to contain bugs or compatibility issues like I’ve seen in the past with various other themes I tried. Consequently, I recently updated my blog’s theme to Twenty Sixteen. To my surprise though, no header image was included (it is included in other WordPress themes) and the top section seemed a little barren.

A quick web search for free WordPress header images turned up a few sites that did have some nice photos, but required you to link back to them which is a bit cumbersome. Fortunately, I realized there was a better way.

Enter Freeimages. This neat site contains almost 400,000 free images – many of them free even for commercial use (if you have ads on your site, it is considered commercial). You can also usually modify the image (so cropping it to fit your theme should be OK), and for the most part not even attribution is necessary! Of course, you should always check the license on each image to be sure.

So in a nutshell, here’s what you need to do:

  1. Search for some pictures you might like for your header
  2. Pick the one you like the most (or several, if you’d like to rotate them)
  3. Download a high-res copy so that WordPress only has to crop it and/or scale it down (rather than scale it up / stretch it)
  4. Make sure you abide by the license
  5. Upload and crop the image (via the WordPress theme customizer)

If you don’t find an image you fancy in that particular site, there are many others you can browse. See if you can spot the image you’re currently seeing at the top of this page ūüėČ

Happy hunting!

Redirecting all URLs to their HTTPS WWW equivalents

Whether you’re a no-www or a yes-www person (you can guess which one I am based on the URL currently present on your address bar ūüôā), one thing is certain – you need to be consistent. This means that if you like www, then yourdomain.com should redirect (301 permanent) to www.yourdomain.com, and if you don’t like it then the redirection needs to go the other way around. Either way, you can’t have people browsing the same page from different domain (with www and without it), it’s just confusing.

In addition, you really should be using SSL everywhere, all the time. Certificates are free, and even if your hosting plan doesn’t support them (as is the case with Azure’s free tier), you can have CloudFlare take care of that for you (and enjoy improved performance and security as a bonus). Once you have SSL up and running, you’d want to redirect all HTTP traffic to HTTPS and enable HTS for increased security.

To recap – once you’ve decided whether you like www or not, and finished setting up SSL, you want your redirections to look like this (assuming yes-www):

  • HTTP://yourdomain.com -> HTTPS://www.yourdomain.com
  • HTTP://www.yourdomain.com -> HTTPS://www.yourdomain.com
  • HTTPS://yourdomain.com -> HTTPS://www.yourdomain.com

In addition, you want HTS headers (Strict_Transport_Security) in place. Fortunately, using the IIS Rewrite module (installed by default on Azure deployments), accomplishing all of the above is a breeze:

<configuration>
<system.webServer>
 <rewrite>
  <rules>
   <rule name="Redirect to www" stopProcessing="true">
    <match url="(.*)" />
    <conditions logicalGrouping="MatchAny">
     <add input="{HTTP_HOST}" pattern="^yourdomain\.com"/>
     <add input="{HTTPS}" pattern="off" ignoreCase="true"/>
    </conditions>
    <action type="Redirect" 
            url="https://www.yourdomain.com/{R:1}" 
            redirectType="Permanent"/>
   </rule>
  </rules>
  <outboundRules>
   <rule name="HSTS" enabled="true">
    <match 
       serverVariable="RESPONSE_Strict_Transport_Security" 
       pattern=".*" />
    <conditions>
     <add input="{HTTPS}" pattern="on" ignoreCase="true" />
    </conditions>
    <action type="Rewrite" value="max-age=31536000" />
   </rule>
  </outboundRules>
 </rewrite>
</system.webServer>
</configuration>

Happy rewriting ūüėČ

Gtk-WARNING **: Locale not supported by C library

I was recently writing a mono application for OSX in MonoDevelop, using Gtk# as the GUI framework. The app seemed to work fine, but it would throw the following error each time it started:

Gtk-WARNING **: Locale not supported by C library

Since I don’t like errors, even if I don’t understand their immediate effect, I did a bit of searching. My search led me to this StackOverflow post,¬†whose answers suggested¬†defining the¬†following environment¬†variables in the terminal:


export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

Indeed, setting those variables in a bash script, prior to the execution of my application, resolved the issue ūüôā

Controlling Roku devices from restricted Windows Phone Devices

Scroll to the bottom if you want the TLDR version of controlling Roku devices from restricted Windows Phone devices

I recently purchased a Roku Streaming Stick after reading some great reviews online, and verifying it comes with Windows Phone support. Hooking it up to the TV was easy enough, and I was quickly able to enjoy streaming content using the included remote.

However, to my dismay I soon realized the remote was defective,¬†intermittently entering a non-responsive state where all key presses would be ignored. I was peeved, of course, but I thought to myself – no big deal, I’ll just use the WP remote app, right?

Wrong. I tried the Roku official app, as well as literally¬†every¬†other Roku remote app on the WP store, to no avail. Most of the apps wouldn’t recognize my Roku device, with the official app saying “no Roku devices were¬†found on your network” and “could not access the information of your Roku device(s)”. When I tried manually connecting to the Roku’s IP (as indicated in the Roku’s network settings page), the official app said it “Failed to connect to the specified device”.¬†The few apps¬†that did seem to “connect” to it¬†could not control it – I would press a button and nothing would happen.

In my despair, I contacted¬†Roku support. The representative¬†quickly determined it was due to my location (Israel), which is not officially supported by Roku. Say what? The device work, its remote (mostly) works, but an app that merely simulates a remote suddenly cares about your location and deliberately fails just for spite? I didn’t buy it.

So I went to the Windows Store on my PC and installed the Roku Windows Store app. It immediately recognized my device and lo and behold Рit was able to control it effortlessly. Motivated by my discovery, I powered on an old Android device and Installed a Roku remote app. Again, it worked right out of the box.

At this point I became¬†very curious.¬†Why were the WP apps failing? It can’t be a coincidence, especially considering the fact that many reviewers confirmed¬†those apps did in fact¬†deliver on their promise and controlled the device. To satisfy my curiosity, I referred to Roku’s REST API documentation. It literally could not be any simpler. Want to press the Home key? This is all you need to do:

curl -d '' http://192.168.1.134:8060/keypress/home

There’s no way¬†all the Roku apps in the WP store got something as¬†simple as this wrong. But seeing how simple the API was, I realized it would be a matter of hours to create a simple WP app that sends these commands and be done with it. I even had a name for my app – RemoRoku. It would be the savior of all Windows Phone Roku users, and I would be forever renowned as the hero of the community.

In reality, I quickly realized why all the other apps were failing. A few XAML tags and a single HttpClient call later, my app revealed curious behavior. The app would work fine on the emulator, but the device itself kept blocking the HTTP requests with 404 Not Found errors.

After a bit of searching on the web¬†I ran into this thread. It mentions a similar phenomena, explaining how “regular” domain requests¬†(e.g. microsoft.com) would work whereas IP requests (e.g. 192.168.1.134) would fail. The author ended¬†up¬†figuring out¬†the root cause of the issue –¬†a missing app capability by the name of¬†privateNetworkClientServer. Unfortunately, setting it in my app’s manifest had no effect (which is not surprising, considering that the docs state¬†it is¬†equivalent on WP to¬†internetClientServer¬†which was already enabled).

At this point I had to assume something was¬†blocked specifically on my phone. Maybe it was the specific hardware/software combination (Lumia 920, WP Denim), or more probably some enterprise Mobile Device Management (MDM) policy set by my workplace (I do work for Microsoft, after all ).¬†Out of curiosity, I looked for such a policy in the WP MDM docs and couldn’t find one, but perhaps it is an implicit side effect of some other policy. It’s possible that¬†the upcoming Windows 10 update will change things, but at this point in time it doesn’t seem¬†like there’s anything I can do about it on my end.

I was undeterred, however. As the saying goes –¬†If the mountain won’t come to Muhammad, then Muhammad must go to the mountain. Windows Phone¬†wants a “real” domain and not and IP? I’ll give him one, by rewiring the DNSMasq rules¬†in my DD-WRT router to redirect a made-up domain of my choice to the Roku’s IP. Using a rule as simple as

address=/ohadroku.com/192.168.1.134

I was able to trick the HttpClient by sending my requests to ohadroku.com, and it worked from the device as well. I then went back to one of the Roku apps that allows you to use a domain name to identify the Roku device (the official one forces you to use an IP address) and typed in my made-up domain. Brimming in anticipation, I clicked the Home button on the WP remote app and sure enough Рthe call went through to the Roku device, which executed the command flawlessly.

If your router doesn’t support¬†domain redirection, you can use a¬†domain you own to do a similar trick.¬†For example, I could have¬†simply added a DNS A record mapping¬†roku.ohadsoft.com¬†to¬†the router’s IP and it would have worked the same. This might be confusing since the router’s IP is not an internet-accessible address, but inside your private network it doesn’t matter. The app would resolve roku.ohadsoft.com¬†(using¬†an actual DNS server) to the router’s (internal) IP address, and¬†since the app is working from inside the network as well, the requests would¬†work.

This spelled the end of¬†RemoRoku before it was even born, and with it my inevitable rise to fame and fortune.¬†I guess you can’t¬†win them all!

The DNSMasq rule¬†did have one unfortunate side¬†effect though – it seems that Plex secure connections require additional configuration¬†in order to work alongside it, and that configuration is not supported on DD-WRT without compromising security or switching a DNS provider¬†(which I did not feel like doing at 4 AM). So¬†I just approved insecure Plex connections inside my private network – I trust my WPA2 protection so¬†I won’t¬†lose much sleep over this.

TLDR

  1. Redirect a made-up domain of your choice to your Roku’s IP address (you can find it in¬†its network settings). This must be done at the router / DNS¬†level, so look for instructions depending on your setup.¬†In DD-WRT you add something like this to the Services -> Additional DNSMasq Options box:¬†address=/ohadroku.com/192.168.1.134. For more information see this article.
  2. Go to the Windows Phone Store and download a Roku remote app that¬†allows you to specify the Roku’s address using a domain name and use your made-up domain. I used Remote 4 Roku Free.
  3. Enjoy your functioning WP Roku Remote!