Rename Perl6 to Perly

As a complete outside spectator of The Perl6 naming disaster I have proposed a rename to PerlyGate to keep conotations to both old Perl and all the religious mumbo jumbo Perl developers are so fond of. But my suggestion seemed to go unnoticed, might be due to the fact this blog is not read by anyone.
But I give it another try. Why not PerlySix, which keeps the reference to Perl6 and at the same time is a new name separated from Perl. Or just Perly a new name keeping the reference to Perl. Both Perly and PerlySix avoids the version number confusion with Perl.

Do the rename of Perl6, and version update Perl to Perl 7, bumping over the unfortunate Perl 6.
Perly can keep the version number 6 or start from 1 both options sounds good to me. 

Update: As I understand Perl6 is now renamed to Raku. To me it sounds very 'pale', no flair whatsoever, but I'm not an native English speaker, so what do I know. Anyway it is the language that counts, not the name.

Data Scientist

For some years now I have seen the new profession "Data Scientist" appear more and more frequently in articles, papers and job advertisements. It seems like every employer want to hire Data Scientists. Lately I have begun to think "How cool wouldn't it be to be a Data Scientist", just the word Scientist has a certain aura around it.Yes I want to become a Data Scientist, so I Googled "what is a data scientist". A lot of hits, and they all replied about the same, IT professional who can help the business to analyse the business (this is one of the best of those I found, go directly to the video clip by Kirk Borne, PhD, Principal Data Scientist at Booz Allen Hamilton. I'm a bit bored at the "this is not an IT function, this is a business function", the weight is definitely  on the IT side.).

The result was sort of an anticlimax, Data Scientist seems to be what I have done part time more or less my entire career. But I have given myself more humble titles, report writer, programmer, data analyst, BI developer, it is basically the same as the more gaudy "Data Scientist". Of course the methods have changed over time in the old cobol paper days, the options were more limited, but the core "help the business find and analyse the data" is very much the same. I have now upgraded myself with the flamboyant title Data Scientist.

I can hear some people say "but you have never understood the business". I have created an MRP system and an BI system, how can you do that without understand the business? When I left the company the first time the IT manager told me "you are the only one in the IT department who fully understand the material flow in the company". That was not really the case, I understood the material flow well in the business area producing mine equipment, for the other business areas my knowledge was more superficial. Nevertheless I think I qualify for the Data Scientist title😎

 

Email automation, C# and PowerShell

Some time ago I did an Email automation application in my Data Warehouse. In April this year I got a

request from “the business”, ‘can you help me develop a macro that creates PDFs’? ‘Uh?’ I replied.

The colleague had a very large Excel workbook with +hundred sheets each of them should be separate PDF files. I’m really bad at Excel I have problems with auto sums, never had written a macro I only done some visual basic scripts. But I promised to look at it and then I forgot all about it. Imay my colleague asked if I had looked at the Excel, I promised to look at it which I did. I wondered why my college wanted PDFs is that not more to it? And then I forgot about it. In June my colleague asked again if I had had time to look at her Excel, now I felt i had to do something, so I asked if the PDFs was all she wanted, can you explain what the source for this Excel is and what you are going to do with the PDFs.. I got a long story about compile material for invoices and create this multi-sheet Excel,create PDFs and send them to up to six recipients of which she only had the names not email addresses. As always when you have an old manual process inherited from person to person

explained to you it sounds complex. I did not understand much about it, but I understood she had a lot of mails to send to a lot of names/recipients.

This was more than I could do with an Excel macro.

I had a large multi-sheet  Excel, each sheet should be sent to a company, and the recipients  

in another Excel Sheet with six columns and one row per receiving company..

First I created a PowerShell script splitting the multi-sheet Excel into separate Excels. I asked if I could attach

an excel to each mail instead of cut and paste a PDF into the mail body as they did. This was one of the odd  tasks in this manual process, this was just done as it was the way they always had done this. I was told this was actually much better with Excel attachments, since these invoices were no legal invoices, just notifications to subsidiaries within the Enterprise.

Next I took care to the translations of recipients names into emailaddresses. I created a C# program

listing the Global Address List, with it I should be able to translate any Enterprise Employee name into

an Email address, of course I was wrong, I missed about 30% of the recipients, it turned out not all subsidiaries was incorporated into the Enterprise mailsystem. I then listed all users in the Active Directory, this time I used PS scripting, after the C# adventure I decided to go for PS scripts, it is much faster to develop and fast enough for this kind of automation. Unfortunately I still missed about 10% of the recipients. For those I created a list manually and also accepted email addresses directly in the recipients list. I had to extract data from two ‘got-it-all’ sources and still had to add a manual source and allow for name and email address specifications of recipients. If I had one source for all recipients I would have saved quite some time, only the C# program extracting GAL took me the better part of a Sunday.

I was given a static mail body to be included in the mails. So the last thing I did was a PS script for the mail

automation. PS scripting is actually very nice for Email automation, just for the hell of it I added variable substitution for the mail template so I could personalize the mails.

Doing this Email automation exercise led me to some conclusions:

PowerShell scripting is very good for automation of manual tasks

Good H/R master data is essential for automation of tasks where people/employees are  involved.

Automation is fun. having done the output part of a manual process, I’m looking forward to take a stab at the input part.

In the next post I show some code:)

Recruitment process out of control

The recruitment process has gone haywire or rather been kidnapped by HR. These days when you apply for a new job you do not only send in the CV and then eventually be selected for interviews by HR and recruiting manager and maybe a subject matter test of some kind. No these days you first have to pass a HR test over the Internet most like created by a hired recruitment company often evaluated by a so called robot. These tests are supposed to measure your IQ and ability to solve complicated problems. I have come across a few such test and they look more like bullshit than anything else in my eyes. You have very little time for each question e.g. the instruction take longer time to read (for a dyslectic) than to solve/reply the question. That is a test for speed reading not for what I think the question is intended to answer. And what does the ability to fill in the missing picture in a Sudoku matrix has to do with the ability to solve complex problems? One day of training on these tests improves your score a lot. Do that day increased you IQ and your ability to solve complex problems? I doubt. 
I am confident this new sifting test process is more harmful than helpful.
I recently heard this true story about a very distinguished science professor who had founded a  research company and let a recruitment company manage the recruitment process. To his disappointment his best PhD student did not apply, so he called him up and asked 'why didn't you apply?'. 'I didn't pass the test' was the reply. The professor called up the recruitment company and told them 'you have turned down a world leading expert' and requested to see the tests and all result. He then took over the recruitment process and hired the PhD who failed the test.     

In memoriam

In some posts I have written about colleagues who have left for new challenges outside the company. This post is different, last week Carina passed away after a short period of severe illness. We did not work together, but we were colleagues, when I was an IT manager she was employed as a secretary. Later when I moved to the HQ she worked as an IT admin and the last years a communicator for the corporate IT  in the same office as I work. I who used to came in first in the morning and leave last in the evening was second when Carina joined us. She was dedicated and hard working like few, but that is not what she is remembered for, it is her positive attitude always cheerful and friendly, we always had a a small chat before the other came in. When she occasionally left before me, I used to say "Working part time now, are we?" And we laughed a bit about that. There is no more "Good morning Lars!" when I arrive at the office. It is funny I did not realized how much that actually meant to me until she become ill just a few months ago. There is a big empty space in the office, farewell Carina my thoughts are with you and your family.

OData: Request failed (404)

I'm trying to get data from SharePoint via an Odata query to Power BI using the Odata connector. My query is perfectly legal in a browser (or Powershell) but in Power BI the query results in a 404 return code. Googling about the error the best reply I get is:
the above error could occur when OData URL string is large or large number of columns are pulled into Power BI Desktop. Please check your URL and try to reduce columns, then check if the error goes away. 
I cannot find any limits or actually anything that tells me what the f-ng error is. I reduce my request to a minimum and I still got the same 404 responce.
It seems the only Odata I successfully process is a simple get one list  Odata request. So I download needed lists one by one and the by clicking around manage to join the lists in Power BI and find this code in the advanced editor:

let    Source = SharePoint.Tables("https://thecompanysharepoint/metadata/application", [ApiVersion = 15]),    #"cffb758e-476c-4673-84f5-7be931f86eb2" = Source{[Id="cffb758e-476c-4673-84f5-7be931f86eb2"]}[Items],    #"Renamed Columns" = Table.RenameColumns(#"cffb758e-476c-4673-84f5-7be931f86eb2",{{"ID", "ID.1"}}),    #"Merged Queries" = Table.NestedJoin(#"Renamed Columns",{"Currency CodeId"},ISOCurrencyCode,{"Id"},"ISOCurrencyCode" ,JoinKind.LeftOuter),    #"Expanded ISOCurrencyCode" = Table.ExpandTableColumn(#"Merged Queries", "ISOCurrencyCode", {"ContentTypeID" , "CurrencyCode", "CurrencyName", "CurrencyNumCode", "CurrencyISOEdition", "CurrencyStatusValue", "CurrencyComment" , "CurrencyAC_Reporting", "Id", "ContentType", "Modified", "Created", "CreatedById", "ModifiedById", "Owshiddenversion", "Version" , "Path", "CurrencyStatus", "CreatedBy", "ModifiedBy", "Attachments"}, {"ISOCurrencyCode.ContentTypeID", "ISOCurrencyCode.CurrencyCode", "ISOCurrencyCode.CurrencyName", "ISOCurrencyCode.CurrencyNumCode", "ISOCurrencyCode.CurrencyISOEdition" , "ISOCurrencyCode.CurrencyStatusValue", "ISOCurrencyCode.CurrencyComment", "ISOCurrencyCode.CurrencyAC_Reporting", "ISOCurrencyCode.Id", "ISOCurrencyCode.ContentType", "ISOCurrencyCode.Modified" , "ISOCurrencyCode.Created", "ISOCurrencyCode.CreatedById", "ISOCurrencyCode.ModifiedById", "ISOCurrencyCode.Owshiddenversion" , "ISOCurrencyCode.Version", "ISOCurrencyCode.Path", "ISOCurrencyCode.CurrencyStatus", "ISOCurrencyCode.CreatedBy" , "ISOCurrencyCode.ModifiedBy", "ISOCurrencyCode.Attachments"}),    #"Filtered Rows" = Table.SelectRows(#"Expanded ISOCurrencyCode", each ([Month] = 2) and ([Year] = "2016")) in    #"Filtered Rows"

I modified this code with great care to get a result I'm happy with since I cannot find any reference or guidance at all for this scripting language which I even do not know the name of.

Anyway now it works, and I got the requested result.

Sisyphus

SharePoint woes again. I’m trying to download some SharePoint lists using CSOM. I find lots of examples

nothing works, first I did the same stupid mistake I always do trying to access SharePoint, even though

you are searching for CSOM examples most hits you get are SSOM examples. After I failed with one SSOM I found a promising example, but it did not work for me.

The powershell script example loaded two dlls. I probably had an invalid version of some SharePoint library

and I could not find one that worked. This time I learned not only do I have choose between SSOM and CSOM,

but it looks like there is onprem and an office365 versons of CSOM or some libraries. I do not know what my

problems really are and I get very little help from searching the web. None of the suggested solutions

for my problem changed anything, the error remained I had my code unsaved in Powershell ISE when my PC

all of a sudden restarted, nothing was saved, now this really does not matter much since the code I worked

with was a dead end. For some reason working with SharePoint reminds me of Sisyphus.