News Group Posts

tr vs. sed url-encoded

From: "Brian K. White" <brian@aljex.com>
Subject: Re: tr syntax
Date: 27 Apr 2005 22:01:01 -0400
Message-ID: <046501c54b96$18014080$6500000a@venti> 
References: <FMTbe.8686$J12.2080@newssvr14.news.prodigy.com> 


----- Original Message ----- 
From: "E Arredondo" <atk@sbcglobal.net>
Newsgroups: comp.unix.sco.misc
Sent: Wednesday, April 27, 2005 5:48 PM
Subject: tr syntax

> Do you know the right syntax for the TR command to strip characters from a 
> file ?
>
>
> my file looks like this :
>
> rfq=ADDRFQ%7ER-050427%3A033325%7ECARQUEST%7E9%7E1.1%7E2005-04-27T12%3A33%3A25.00
> 0-07%3A00%0D%0ARFQ%7ER-050427%3A033325%7ER-050427%3A033325%7E2005-04-27T12%3A33%
> 3A25.000-07%3A00%7E0%7E0%7E57603%7Etest%7E654654+test+ave%7E%7E%7Esanta+ana%7ECA
> %7E92704%7EUS%7E%28714%29+850-1544%7E32423%7ENew+York+DC%7E32+Apple+Lane%7E%7E%7
> ENew+York%7ENY%7E10010%7EUS%7E%28212%29+212-2122%7E%0D%0ALINE%7ER-050427%3A03332
> 5%7E1%7ER-050427%3A033325%7EDF01%7EDF01%7E1.0%7EEA%7EFREGHT%7E%0D%0A
>
>
> and I want to change all the   %7E   for an  ~
>
> and I've tried
>
> cat file.txt | tr "%7E" "~"    but only changes the % for an  ~  and 
> leaves the 7E.
>
> Then I want to change the %0D%0A  for a Line feed or carriage return.
>
> cat file.txt | tr "%0D%0A" "/012"
>
>
> Thanks

tr can not do that job, for that you want sed, or....

This is a good example of what we mean when we sometimes tell people "tell 
us what you really want" instead of trying to abstract out the part you 
think you need solved out of the real whole problem.

This appears to be url-encoded content, you may be interested to know there 
are utilities and code snippets in various languages already designed just 
for the purpose of url en/de-coding. There are a lot of other codes besides 
the couple you asked about, and you should really be prepared to decode any 
that might appear in the data, including ones that you haven't just happened 
to encounter yet.

I found a couple of snippets of c code that do this and made them into bare 
bones stand-alone executables.
http://www.aljex.com/bkw/sco/index.html#urlenc

Unfortunately, your data also includes "+" in place of spaces, which is 
another common url hack, and unfortunately, it wouldn't be safe/sane in my 
opinion to wholesale replace all + with " " (which tr can do) because surely 
a "+" is a legitimate character that might appear in the data?

The spaces should have been submitted to the cgi as %20 instead of + in the 
first place. By the time you recieve the data above, it's too late for you 
to be able to reliably convert the +'s back to spaces without also 
converting +'s that were supposed to actually be +'s in the data.

Here's a command to use the binary above and tr to decode all the url codes 
and convert all +'s to spaces and just hope no + ever comes in as data.
deurl "$rfq" |tr "+" " "

My guess is you are ultimately getting this data into filepro. You could 
leave off the tr and use xlate in filepro almost as easily. In that case, 
you could decide to only xlate certain fields, leaving the possibility of 
real data +'s in some fields. The tr command will convert every + in the 
record and has no concept of fields.

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani
 

Comments?

Jump to Comments

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them. I resell or can earn commissions from the sale of some of these items. Links within these pages may be affiliate links that pay me for referring you to them. That's mostly insignificant amounts of money; whenever it is not I have made my relationship plain. I also may own stock in companies mentioned here. If you have any question, please do feel free to contact me.

Specific links that take you to pages that allow you to purchase the item I reviewed are very likely to pay me a commission. Many of the books I review were given to me by the publishers specifically for the purpose of writing a review. These gifts and referral fees do not affect my opinions; I often give bad reviews anyway.

We use Google third-party advertising companies to serve ads when you visit our website. These companies may use information (not including your name, address, email address, or telephone number) about your visits to this and other websites in order to provide advertisements about goods and services of interest to you. If you would like more information about this practice and to know your choices about not having this information used by these companies, click here.