Friday, March 18, 2022

Run powershell specific function from task scheduler

Executable is powershell.exe. Arguments:

-command "& { . "c:\location\to\script.ps1"; my_function_name }"

Powershell speed hacks

Powershell can be painfully slow when dealing with larger arrays, reading files and listing large directories. Here are some workarounds.

Arrays
Slow:

$myarray = @()
foreach ($x in $y) {
  $myarray += $x
}

Much faster is working with an arraylist:

$myarray = [System.Collections.ArrayList]@()
foreach ($x in $y) {
  $null = $procarray.Add($x)
}

Reading files
Slow:

get-content $filename

Fast:

([System.IO.File]::ReadAllLines($filename))

Listing large directories
Slow:

$items = get-item "\\server\share\*.csv" | sort LastWriteTime

The fastest workaround i’ve been able to find is actually using a dos prompt. Use dir switches for sorting purposes.
Note: dir returns just text, while get-items returns objects with all sorts of properties. It depends on your use case whether this hack is actually usable or not.

$items = cmd /r dir "\\server\share\*.csv" /OD /B

Wednesday, February 10, 2021

Azure/O365/Teams authentication and monitoring bash curl scripts

Authorize for teams.
Replace YOUR_TENANT_ID, YOUR_EMAIL and YOUR_PASSWORD.
Use one of these client_id’s, depending on your usecase.
1fec8e78-bce4-4aaf-ab1b-5451cc387264 (Teams mobile/desktop application)
5e3ce6c0-2b1f-4285-8d4b-75ee78787346 (Teams web application)

auth.sh:

#!/bin/bash

curl -s -X POST https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/token \
-c cookies.txt \
-o auth.blob \
-F grant_type=password \
-F resource=https://teams.microsoft.com/ \
-F client_id=1fec8e78-bce4-4aaf-ab1b-5451cc387264 \
-F username=YOUR_EMAIL \
-F password=YOUR_PASSWORD

This will save your bearer token, amongst others, to auth.blob in a json object.

Because the bearer token is only valid for a certain period of time, you’ll need to refresh it. Here’s how. You’ll need ‘jq’ installed to decompose the json object.
refresh.sh:

#!/bin/bash

REFRESHTOKEN=`cat auth.blob | jq ".refresh_token" | sed 's/"//g'`

curl -s -X POST https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/token \
-c cookies.txt \
-o auth.blob \
-F grant_type=refresh_token \
-F resource=https://teams.microsoft.com/ \
-F client_id=1fec8e78-bce4-4aaf-ab1b-5451cc387264 \
-F refresh_token=$REFRESHTOKEN

In the script you can keep repeating actions, but in order to keep your token active, you can use the following piece of code:

if [ -f "auth.blob" ]; then
  EXPIRES=`cat auth.blob | jq ".expires_on" | sed 's/"//g'`
  NOW=`date +%s`
  TTL=`expr $EXPIRES - $NOW`
  if [ $TTL -lt 60 ]; then
    echo "time for a refresh!"
    ./refresh.sh
  fi
else
  echo "no previous auth present!"
  ./auth.sh
  EXPIRES=`cat auth.blob | jq ".expires_on" | sed 's/"//g'`
  NOW=`date +%s`
  TTL=`expr $EXPIRES - $NOW`
fi

Now you can do the cool stuff like query your calendar or whatever:

#!/bin/bash

BEARER=`cat auth.blob | jq ".access_token" | sed 's/"//g'`
curl -s --write-out "%{http_code}|%{time_total}n" -o bla.txt "https://teams.microsoft.com/api/mt/emea/beta/me/calendarEvents?StartDate=2021-02-07T23:00:00.000Z&EndDate=2021-02-14T23:00:00.000Z" \
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Teams/1.3.00.30866 Chrome/80.0.3987.165 Electron/8.5.1 Safari/537.36" \
-H "authorization: Bearer $BEARER"

Or verify your local timezone:

#!/bin/bash

BEARER=`cat auth.blob | jq ".access_token" | sed 's/"//g'`

date "+%Y.%m.%e %T %N"
curl -v 'https://teams.microsoft.com/api/mt/part/emea-03/beta/me/calendarEvents/timeZoneSettingsWithOffset?timezone=Europe%2FAmsterdam' \
-H "authorization: Bearer $BEARER" \
-H 'authority: teams.microsoft.com'
echo ""
date "+%Y.%m.%e %T %N"

Wednesday, June 24, 2020

iptables log specific connections

Example how to allow certain known connections (e.g. unifi accesspoints) and log unknown connection attempts.
This is done by adding a chain called LOGDROP, append packets that match the criteria (tcp/8080) to that chain, log the packets and drop them.

iptables:

#!/bin/bash

AP01="192.168.0.1"
AP02="192.168.0.2"
AP03="192.168.0.3"

# Resetting ...
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -F
iptables -X

# Setting default policy on incoming traffic
iptables -P INPUT DROP                                                  # DENY INCOMING CONNECTIONS
iptables -P FORWARD DROP                                                # THIS IS NOT A ROUTER

# allowed accesspoints
iptables -A INPUT -p tcp --dport 8080 -s $AP01 -j ACCEPT                # UNIFI - AP01
iptables -A INPUT -p udp --dport 3478 -s $AP01 -j ACCEPT
iptables -A INPUT -p tcp --dport 8080 -s $AP02 -j ACCEPT                # UNIFI - AP02
iptables -A INPUT -p udp --dport 3478 -s $AP02 -j ACCEPT
iptables -A INPUT -p tcp --dport 8080 -s $AP03 -j ACCEPT                # UNIFI - AP03
iptables -A INPUT -p udp --dport 3478 -s $AP03 -j ACCEPT
# log AP connections that aren't allowed
iptables -N LOGDROP
iptables -A INPUT -p tcp --dport 8080 -j LOGDROP
iptables -A LOGDROP -j LOG --log-prefix "IPTables-Dropped: " --log-level 7
iptables -A LOGDROP -j DROP

# Make persistent
iptables-save >/etc/iptables/rules.v4

Create a file in /etc/rsyslog.d/ called “30-unifi-accesspoints.conf” with the following content:

:msg,contains,"IPTables-Dropped: " /var/log/unifi_accesspoints.log

and restart rsyslog

Monday, August 25, 2014

Areca and s.m.a.r.t. monitoring

After swapping a couple of defective harddisks, i was wondering why i never got a predictive failure from my Areca controller.
The weird thing is: the logging shows warnings:

2014-08-24 23:15:37  IDE Channel #08  Reading Error
2014-08-24 23:15:28  IDE Channel #08  Reading Error
2014-08-24 23:15:19  IDE Channel #08  Reading Error
2014-08-24 23:15:10  IDE Channel #08  Reading Error

However.. the controller doesn’t seem to do anything with the s.m.a.r.t. values.
Here’s a script you might want to use as a base to get your monitoring up and running.

#!/bin/bash

CLI="/path/to/cli32"

NR_OF_PORTS=`$CLI disk info | wc -l`
# subtract 4 to get rid of the formatting and determine the real number of disks
NR_OF_PORTS=`expr $NR_OF_PORTS - 4`
echo "Controller has $NR_OF_PORTS ports"

for (( i=1; i<=$NR_OF_PORTS; i++ ))
do
  RELOC_SECT=`$CLI disk smart drv=$i | grep "Reallocated Sector Count" | awk '{print $9}'`
  if [ -z "$RELOC_SECT" ]; then
    echo "Port $i = No Disk"
  else
    echo "Port $i = $RELOC_SECT"
  fi
done

Thursday, April 17, 2014

Powershell date conversion

By default, powershell uses your regional settings. So when importing data from external files, a simple get-date or typecast to [DateTime] does not always give the correct value.
With the template below, you can interpret any format.

PS> $timeinfo = '12 07 2012 18 02'
PS> $template = 'HH mm yyyy dd MM'
PS> [DateTime]::ParseExact($timeinfo, $template, $null) 
Samstag, 18. Februar 2012 12:07:00

Values can be:

d     Day of month 1-31
dd    Day of month 01-31
ddd   Day of month as abbreviated weekday name
dddd  Weekday name
h     Hour from 1-12
H     Hour from 1-24
hh    Hour from 01-12
HH    Hour from 01-24
m     Minute from 0-59
mm    Minute from 00-59
M     Month from 1-12
MM    Month from 01-12
MMM   Abbreviated Month Name
MMMM  Month name
s     Seconds from 1-60
ss    Seconds from 01-60
t     A or P (for AM or PM)
tt    AM or PM
yy    Year as 2-digit
yyyy  Year as 4-digit
z     Timezone as one digit
zz    Timezone as 2-digit
zzz   Timezone

Friday, December 20, 2013

Batchfile loginscript domain check

ping servername.domain.local -n 1 >NUL
if NOT %ERRORLEVEL%==0 GOTO OFFLINE
  call \\servername.domain.local\share\Extra_Login_Stuff.bat
:OFFLINE

Monday, May 13, 2013

Dump Exchange mailbox permissions

A complete script to first dump all exchange mailboxes to .csv and then enumerate all mailbox permissions.
It uses the Exchange 2010 management shell and Quest’s Active Directory Powershell modules.

Usage:

  • Load the script in the ISE editor.
  • Set the two global parameters
  • Run the script
  • first execute: dump_mailboxes (this wil generate a .csv with all mailboxes)
  • then execuite: dump_all_mailbox_permission (this will generate a second .csv with all permissions. Open in Excel to filter)
echo "-"

$global_ad_domain = "AD.CUSTOMER.LOCAL"
$global_ad_short = "AD"

### Load Modules for Active Directory and Exchange 2010
if (!($QUEST_LOADED))
{
  Add-PSSnapin Quest.ActiveRoles.ADManagement
  Set-QADPSSnapinSettings -DefaultSizeLimit 0

  $logged_on_to = $env:USERDNSDOMAIN
  if (!($logged_on_to -eq "$global_ad_domain"))
  {
    $user = read-host "Enter username in adusername format"
    $pw = read-host "Enter password" -AsSecureString
    connect-QADService -service '$global_ad_domain' -ConnectionAccount $user -ConnectionPassword $pw
  }
  else
  {
    connect-QADService
  }
  
  Set-QADProgressPolicy -ShowProgress $false
  $QUEST_LOADED=$TRUE
  echo "quest loaded"
}

if ($EMS_loaded -eq $NULL)
{
  . 'C:\Program Files\Microsoft\Exchange Server\V14\bin\RemoteExchange.ps1'
  echo "- Exchange Management Shell Loaded"
  Connect-ExchangeServer -auto
  $EMS_loaded = $true
  echo "- Exchange Management Shell Connected"
}

### Functions

function dump_mailboxes
{
  $output_file = "d:\temp\mailboxes.csv"
  echo "Name`tAlias" >$output_file
  # $mailboxes = Get-Mailbox -RecipientTypeDetails SharedMailbox
  $mailboxes = Get-Mailbox -resultsize Unlimited
  foreach ($mailbox in $mailboxes)
  {
    $Name = $mailbox.Name
    $Alias = $mailbox.Alias
    echo "$Name`t$Alias" >>$output_file
  }
}

function dump_all_mailbox_permission
{
  $output_file = "d:\temp\mailbox_permissions.csv"
  $lijst = import-csv -delimiter "`t" d:\temp\mailboxes.csv
  $aantal = $lijst.count
  $teller = 0
  write-host "Aantal functionele mailboxen: $aantal"
  echo "Mailbox`tAuthType`tGroup`tSam`tType" >$output_file  
  foreach ($regel in $lijst)
  {
    $teller++
    $Alias = $regel.alias
    write-host "$teller / $aantal -> $Alias"
    mailbox_permissions $Alias >>$output_file
  }
}

function mailbox_permissions($mailbox)
{
  if ($perms = get-mailboxpermission -identity "$mailbox" | where {($_.isinherited -eq $false) -and ($_.User -like "$global_ad_short\*")})
  {
    foreach ($perm in $perms)
    {
      $usr = $perm.User.tostring()
      $typeusr = (get-qadobject -identity $usr -DontUseDefaultIncludedProperties).type
      $usr = $usr.replace("$global_ad_short","")
      $rights = $perm.AccessRights
      if ($typeusr -eq "group")
      {
        $members = get-qadgroupmember -identity "$usr"
        foreach ($member in $members)
        {
          $mbmrsam = $member.samaccountname
          echo "$mailbox`t$typeusr`t$usr`t$mbmrsam`t$rights"
        }      
      }
      else
      {
        echo "$mailbox`t$typeusr`t`t$usr`t$rights"
      }
    }
  }
}

echo "-"

Tuesday, November 6, 2012

MemberOf, AllMemberOf, NestedMemberOf

PS D:Usersxxx> (get-qaduser "myaccount").memberof.count
46

PS D:Usersxxx> (get-qaduser "myaccount").allmemberof.count
98

PS D:Usersxxx> (get-qaduser "myaccount").nestedmemberof.count
53

According to: http://msdn.microsof … ibrary/ms677943.aspx: “memberOf does not contain the user’s membership in domain local and global groups in other domains.

Indeed, AllMemberOf shows these groups too (DomainLocal only in my example).

PS D:Usersxxx> $groups = (get-qaduser "myaccount").allmemberof

PS D:Usersxxx> foreach ($group in $groups)
{
  (get-qadgroup $group).GroupScope
}
Global
Global
Global
DomainLocal
Global

Wednesday, September 5, 2012

get size of directories with powershell, the stupid but fast way

All those ways to get the size of directories with powershell are extremely slow. Especially on network shares.
e.g.

$colItems = (Get-ChildItem C:Scripts | Measure-Object -property length -sum)
"{0:N2}" -f ($colItems.sum / 1MB) + " MB"

Currently i’m harvesting through roughly 40TB of data and it’s taking me daaaaaaaaaays!
So i’m in desperate need of something faster.
Then i thought about robocopy. Robocopy gives great statistics. So if i do a “dry-run” (list-only, not really copy), i might get the information i need by parsing the output.

Choice of switches:

  • /b = backup mode. Supposed to give me access to every file
  • /l = list only/dry-run, not really doing the copy
  • /mir = action what you would normally do when you would copy the data. This also dives into all subdirectories.
  • /r:0 = no retries
  • /w:0 = don’t wait on anything
  • /ns /nc /nfl /ndl /njh = no logging of any kind. We only want the summary.

Then we get this piece of code (it could be a lot shorter, but i’m keeping it readable):

function get_size_of_dir_in_bytes_with_robocopy ($directory)
{
  write-host "- $directory" -foreground "GREEN"
  [string]$result = robocopy /b /l /mir "$directory" "c:\whatever" /r:0 /w:0 /ns /nc /nfl /ndl /njh /bytes
  if (!($lastexitcode -eq 16))
  {
    $pos = ($result).indexof("Bytes : ")
    $start = $pos + 8
    $length = $result.length
    $end = $length - $start
    $newstring = ($result).substring($start,$end)
    $newstring = $newstring.trim()
    echo $newstring.split()[0]
  }
  else
  {
    echo "CANNOT ACCESS"
  }
}