I was unable to connect to Mac today.  Not sure why. 

I found this documentation from Xamarin here. 

The document here gave me the solution.
Once I did the following I was able to use the Xamarin Mac Agent (Visual Studio – Tools – iOS – Xamarin Mac Agent) to connect.

Next, test if the ssh client from OpenSSH can connect successfully to the Mac from Windows. One way to install this program is to install

Git for Windows. You can then start a Git Bash command prompt and attempt to ssh in to the Mac with your username and IP address

Once this was permanently added the Max Agent successfully connected. 

By the way I found SSH – I had Git for Windows installed, by opening GIT Shell.