Removing Duplicate Lines from a text or Excel file

Removing duplicate lines on a file is easy. Although, not sure how to do it with vi or notepad++ or crimson editor.

Here’s how to do it in Excel:

1). Open the text file in Excel
2). Create a new macro
3). Cut and paste the procedure below into the macro
4). Run the macro
5). Relish how easy things are with Google

Public Sub DeleteDuplicateRows()
””””””””””””””””””””””””””””””””””””””’
‘ DeleteDuplicateRows
‘ This will delete duplicate records, based on the Active Column. That is,
‘ if the same value is found more than once in the Active Column, all but
‘ the first (lowest row number) will be deleted.

‘ To run the macro, select the entire column you wish to scan for
‘ duplicates, and run this procedure.
”””””””””””””””””””””””””””””””””””””””

Dim R As Long
Dim N As Long
Dim V As Variant
Dim Rng As Range

On Error GoTo EndMacro
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual

Set Rng = Application.Intersect(ActiveSheet.UsedRange, _
ActiveSheet.Columns(ActiveCell.Column))

Application.StatusBar = “Processing Row: ” & Format(Rng.Row, “#,##0”)

N = 0
For R = Rng.Rows.Count To 2 Step -1
If R Mod 500 = 0 Then
Application.StatusBar = “Processing Row: ” & Format(R, “#,##0”)
End If

V = Rng.Cells(R, 1).Value
”””””””””””””””””””””””””””””””””””””’
‘ Note that COUNTIF works oddly with a Variant that is equal to vbNullString.
‘ Rather than pass in the variant, you need to pass in vbNullString explicitly.
”””””””””””””””””””””””””””””””””””””’
If V = vbNullString Then
If Application.WorksheetFunction.CountIf(Rng.Columns(1), vbNullString) > 1 Then
Rng.Rows(R).EntireRow.Delete
N = N + 1
End If
Else
If Application.WorksheetFunction.CountIf(Rng.Columns(1), V) > 1 Then
Rng.Rows(R).EntireRow.Delete
N = N + 1
End If
End If
Next R

EndMacro:

Application.StatusBar = False
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
MsgBox “Duplicate Rows Deleted: ” & CStr(N)

End Sub

This message was sent using PIX-FLIX Messaging service from Verizon Wireless!
To learn how you can snap pictures with your wireless phone visit

www.verizonwireless.com/getitnow/getpix.

To learn how you can record videos with your wireless phone visit www.verizonwireless.com/getitnow/getflix.

To play video messages sent to email, QuickTime® 6.5 or higher is required. Visit www.apple.com/quicktime/download to download the free player or upgrade your existing QuickTime® Player. Note: During the download
process when asked to choose an installation type (Minimum, Recommended or Custom), select Minimum for faster download.

This message was sent using PIX-FLIX Messaging service from Verizon Wireless!
To learn how you can snap pictures with your wireless phone visit

www.verizonwireless.com/getitnow/getpix.

To learn how you can record videos with your wireless phone visit www.verizonwireless.com/getitnow/getflix.

To play video messages sent to email, QuickTime® 6.5 or higher is required. Visit www.apple.com/quicktime/download to download the free player or upgrade your existing QuickTime® Player. Note: During the download
process when asked to choose an installation type (Minimum, Recommended or Custom), select Minimum for faster download.

Yum not working

Yum should work out of the box: but it doesn’t. On CentOS (Linux) you may see the following:

[root@yourservername etc]# yum update apcupsd
Loading “installonlyn” plugin
Setting up Update Process
Setting up repositories
Reading repository metadata in from local files
Could not find update match for apcupsd
No Packages marked for Update/Obsoletion

Add these to “/etc/yum.repos.d/CentOS-Base.repo”:

[extras2]
name=Fedora Extras – $releasever – $basearch
baseurl=http://download.fedora.redhat.com/pub/fedora/linux/extras/6/$basearch/
gpgcheck=0
gpgkey=http://download.fedora.redhat.com/pub/fedora/linux/extras/RPM-GPG-KEY-Fedora-Extras

NOW, try it again:

[root@servername etc]# yum install apcupsd
Loading “installonlyn” plugin
Setting up Install Process
Setting up repositories
extras2 100% |=========================| 1.1 kB 00:00
Reading repository metadata in from local files
primary.xml.gz 100% |=========================| 1.7 MB 00:11
################################################## 5211/5211
Parsing package install arguments
Resolving Dependencies
–> Populating transaction set with selected packages. Please wait.
—> Downloading header for apcupsd to pack into transaction set.
apcupsd-3.12.4-5.fc6.i386 100% |=========================| 15 kB 00:00
—> Package apcupsd.i386 0:3.12.4-5.fc6 set to be updated
–> Running transaction check

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
apcupsd i386 3.12.4-5.fc6 extras2 405 k

Transaction Summary
=============================================================================
Install 1 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

Total download size: 405 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): apcupsd-3.12.4-5.f 100% |=========================| 405 kB 00:02
Running Transaction Test
warning: apcupsd-3.12.4-5.fc6: Header V3 DSA signature: NOKEY, key ID 1ac70ce6
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: apcupsd ######################### [1/1]

Installed: apcupsd.i386 0:3.12.4-5.fc6
Complete!

SSL with Sendmail

Thank to the following web site:

http://www.cromwell-intl.com/unix/sendmail-ssl.html

-Andy

How to make sendmail use SSL/TLS whenever possible

1. Create a certificate directory and go there:
# mkdir /etc/mail/cert
# cd /etc/mail/cert
2. Create a key for the server, giving a new pass phrase when prompted:
# openssl genrsa -des3 -out server.key 1024
# openssl rsa -in server.key -out server.key.open
3. Create a clear-text copy of the key (so it is not pass-phrase-protected), giving the pass phrase when asked:
# openssl req -new -x509 -days 3650 -key server.key.open -out server.crt
4. Answer the X.509v3 questions appropriately.
5. Make the files root-read-only:
# chmod 600 server.*
6. Edit /etc/mail/sendmail.cf and add these lines. You should find commented-out versions of these settings in the file, maybe about a third of the way through it.
O CACertPath=/etc/mail/cert
O CACertFile=/etc/mail/cert/server.crt
O ServerCertFile=/etc/mail/cert/server.crt
O ServerKeyFile=/etc/mail/cert/server.key.open
O ClientKeyFile=/etc/mail/cert/server.crt
7. Restart sendmail:
# /etc/init.d/sendmail restart

Outlook 2003 Not Saving to Sent Items Folder

Outlook 2003 defies intuition. It would make sense that there should be an outbox or a sent items or a “sent” but Microsoft would rather make it really confusing. If you’re connecting through imap and you want to have a specific imap folder say “Sent Items” store your sent mail you need to create a rule:

Tools > Rules > New Rule

Don’t use the template, start from scratch.

What you want to create is a rule that reads like this:

Apply this rule after I send the message. Uses the message form and on this machine only move a copy to Sent Items folder.

-Andy

This message was sent using PIX-FLIX Messaging service from Verizon Wireless!
To learn how you can snap pictures with your wireless phone visit

www.verizonwireless.com/getitnow/getpix.

To learn how you can record videos with your wireless phone visit www.verizonwireless.com/getitnow/getflix.

To play video messages sent to email, QuickTime® 6.5 or higher is required. Visit www.apple.com/quicktime/download to download the free player or upgrade your existing QuickTime® Player. Note: During the download
process when asked to choose an installation type (Minimum, Recommended or Custom), select Minimum for faster download.

This message was sent using PIX-FLIX Messaging service from Verizon Wireless!
To learn how you can snap pictures with your wireless phone visit

www.verizonwireless.com/getitnow/getpix.

To learn how you can record videos with your wireless phone visit www.verizonwireless.com/getitnow/getflix.

To play video messages sent to email, QuickTime® 6.5 or higher is required. Visit www.apple.com/quicktime/download to download the free player or upgrade your existing QuickTime® Player. Note: During the download
process when asked to choose an installation type (Minimum, Recommended or Custom), select Minimum for faster download.

8am

This message was sent using PIX-FLIX Messaging service from Verizon Wireless!
To learn how you can snap pictures with your wireless phone visit

www.verizonwireless.com/getitnow/getpix.

To learn how you can record videos with your wireless phone visit www.verizonwireless.com/getitnow/getflix.

To play video messages sent to email, QuickTime� 6.5 or higher is required. Visit www.apple.com/quicktime/download to download the free player or upgrade your existing QuickTime� Player. Note: During the download
process when asked to choose an installation type (Minimum, Recommended or Custom), select Minimum for faster download.

5pm

This message was sent using PIX-FLIX Messaging service from Verizon Wireless!
To learn how you can snap pictures with your wireless phone visit

www.verizonwireless.com/getitnow/getpix.

To learn how you can record videos with your wireless phone visit www.verizonwireless.com/getitnow/getflix.

To play video messages sent to email, QuickTime� 6.5 or higher is required. Visit www.apple.com/quicktime/download to download the free player or upgrade your existing QuickTime� Player. Note: During the download
process when asked to choose an installation type (Minimum, Recommended or Custom), select Minimum for faster download.