I don’t know about the other people but to me the scariest UNIX command is by far sed. Today I found myself in a situation where I found no other way to solve my problem but to use sed… and it worked… and was not all that scary 🙂
I needed to replace a bunch of strings in a bunch of files in a bunch of subdirectories. Yo know… the kind of fun people are having when moving around and re-organizing things that depend on tons of config files…
So I had to replace all strings which start with ../ to ../../ – in other words I was moving one of my directories deeper into the tree and now to access some paths which were 1 level above we need to go 2 levels up.
I tried the standard ‘replace’ command, some gui tool, some bash voodoo of my own and nothing worked.
Finally I resorted to sed and the line that did the job for me was:
$ find ./ -type f -exec sed -i -r s'/^\.\.\//..\/..\//' {} \;
The GREEN part is the “from” string, the BLUE part is the “to” string and the RED chars are part of the sed ‘search’ command syntax…
Hi,
Can you clarify the significance of each of the characters you used.
1) ^ ( In the FROM )
2) In the from you escaped each of the DOT(.) as well as the directory separator (/) with a backslash (\).
But in the TO, you escaped ONLY the directory separator (/) with a backslash (\).
Why?
Please let me know, before I start to bang my head on the desk. I am desperate to know this..
Thanks
Hi James
The ^ in the search string means “lead space”, i.e. any tabs, spaces ot any other chars considered as space. This is there to tell sed to skipp past all such chars.
In the search (“FROM”) string the . has a special meaning and means “look for any character” that’s why it is escaped – to tell sed that we want it to look for a real dot, not just any character. In the rewrite string it does not have this special meaning (you can’t tell sed “ok write ANY chaaracter here”, right?). That’s why we don’t need to escape it in the TO part.
The backslash always needs to be escaped.
You may also look here for more detailed info:
https://www.gnu.org/software/sed/manual/html_node/Regular-Expressions.html
And this is how you can use sed to break long lines in a file (insert newlines at specific locations). In my case i wanted to make a DB export file (.sql) more readable:
cat file.sql | sed -e $'s/),/\),\\n/g' > file2.sqlThe key part here is that the EOL char here is given to sed as \\n, not just \n
Another useful tool when dealing with special characters such as newline, spaces, etc is
tr…Replacing newline with space:
tr '\n' ' '<file.in>file.outRemoving spaces from a file:
tr -d ' '<file.in>file.outMore details on the subject here:
http://askubuntu.com/questions/537956/sed-to-delete-blank-spaces