<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Life in Pain &#187; technical</title>
	<atom:link href="http://blog.ericw.org/tag/technical/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.ericw.org</link>
	<description>live, love, hurt</description>
	<lastBuildDate>Wed, 18 May 2011 02:37:26 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
		<item>
		<title>Deploying Rack Apps With Cherokee and RVM</title>
		<link>http://blog.ericw.org/2011/02/rack-cherokee-rvm/</link>
		<comments>http://blog.ericw.org/2011/02/rack-cherokee-rvm/#comments</comments>
		<pubDate>Wed, 23 Feb 2011 15:22:06 +0000</pubDate>
		<dc:creator>Eric Will</dc:creator>
				<category><![CDATA[technology]]></category>
		<category><![CDATA[cherokee]]></category>
		<category><![CDATA[rack]]></category>
		<category><![CDATA[rails]]></category>
		<category><![CDATA[ruby]]></category>
		<category><![CDATA[rvm]]></category>
		<category><![CDATA[technical]]></category>
		<category><![CDATA[thin]]></category>
		<category><![CDATA[unicorn]]></category>

		<guid isPermaLink="false">http://blog.ericw.org/?p=140</guid>
		<description><![CDATA[A very long effort to document my rails/rack deployment using cherokee and RVM.</p]]></description>
			<content:encoded><![CDATA[<p>I recently spent three days trying to get <a href="http://www.cherokee-project.com/">cherokee</a>-1.0.19 (the version in <a href="http://www.freebsd.org/">FreeBSD</a> ports as of this writing) to serve up a variety of custom web apps (mostly <a href="http://www.rubyonrails.org/">rails</a>-2.3.8) partitioned and <a href="http://rvm.beginrescueend.com/">RVM</a>. I had pretty much zero success Googling for help during this exercise so I thought I might compile my experience here for future Googlers.</p>

<p>Before this I had them deployed using <a href="http://httpd.apache.org/">apache</a>22 and <a href="http://www.modrails.com/">Phusion Passenger</a> (without <a href="http://rvm.beginrescueend.com/">RVM</a>). This set up wasn&#8217;t working well for me. I&#8217;m not saying it&#8217;s the wrong way to go (I&#8217;m also not going to say a zillion rails users can&#8217;t be wrong) but it wasn&#8217;t performing well for me and it wasn&#8217;t really my style. I&#8217;ve been using apache forever, and piling more and more crap on top of it was just getting frustrating. It really just wasn&#8217;t built for this.</p>

<p>So I start to hear about all these young httpds like <a href="http://www.lighttpd.net/">lighttpd</a>, <a href="http://nginx.org/en/">nginx</a>, and <a href="http://www.cherokee-project.com/">cherokee</a>. I poked around a bit, and I decided cherokee&#8217;s graphical configuration interface sounded nice for a change. At this point I should inform you that I did not at all download or look much into <a href="http://www.lighttpd.net/">lighttpd</a> or <a href="http://nginx.org/en/">nginx</a>. They could be just as good for this, but that&#8217;s another blog entry on another blog.</p>

<p><span id="more-140"></span></p>

<h3>Mongrel or Unicorn or Thin or WEBrick or&#8230;</h3>

<p>One of the first things you need to decide is which back end you want to run. Cherokee needs to know about these processes as this is how it&#8217;ll speak to your app (see the cherokee section below). By default, rails (and almost any rack app) uses WEBrick. WEBrick is great for development, but it&#8217;s pure ruby and pretty slow. I don&#8217;t think you want it serving up your production sites. Historically, a mongrel cluster has been the method of choice for deploying rails. However, mongrel doesn&#8217;t run on ruby-1.9 which is what I use to deploy. <a href="http://code.macournoyer.com/thin/">Thin</a> is a great little server that combines mongrel&#8217;s parser with EventMachine (a fast event-based code system) with rack. Thin can run just about any kind of rack app (rack, rails2, rails3, <a href="http://ramaze.net/">ramaze</a>, probably more) and is really easy to use. However, cherokee isn&#8217;t great at maintaining a thin cluster for you. If one of your instances dies, you&#8217;re going to end up handing someone a 502 error at least once. I personally decided to go with unicorn. <a href="http://unicorn.bogomips.org/">Unicorn</a> can&#8217;t really handle being a front-facing server because it has no KeepAlive support. It acknowledges this, and expects you to use nginx (or in our case, cherokee) in front of it. The great thing about unicorn is that it spawns a management process and then a cluster of workers. The management process does the listening, and decides which worker is best suited to handle your request. If any of the workers die, or need to be killed, the management process will restart them. This means you only need one information source in cherokee (since it manages its own workers it only needs one port, whereas something like thin will need a separate port and thus a separate information source per server in a cluster). I ended up going with unicorn for my big production rails apps with 2-4 workers, and a single thin instance for small <a href="http://www.sinatrarb.com/">sinatra</a> apps that don&#8217;t get a lot of load.</p>

<p>When it comes down to it, this decision is up to you. If you like thin be prepared to juggle as many information sources in cherokee as you have thin servers per cluster. Also be prepared for 502 errors while cherokee spawns thin if it&#8217;s not running. Once spawned, thin will stay running even if cherokee exits (not the case with unicorn, which will exit if cherokee exits or restarts). I don&#8217;t have personal experience with mongrel, but it probably goes something along the same path as thin. WEBrick&#8230; well&#8230; I guess if that&#8217;s how you roll.</p>

<h3>Cherokee and RVM</h3>

<p>Cherokee was the relatively easy part. Aside from the fact that the neat wizards it uses to add virtual servers don&#8217;t usually work right (it seems to assume you&#8217;re using rails-2.2 from the ancient times), the process is pretty simple. You can use the rails wizard to get started. After that, go to the information sources it creates. Here is where you need to tweak. It creates three by default. Depending on which back end you decided on earlier, you&#8217;ll have to tweak these to your pleasure. Most important is the spawning command. This is the command cherokee executes if it can&#8217;t find the interpreter running where it&#8217;s supposed to be. First, however, each information source has to have an environment. You can inherit the environment from cherokee (probably not what you want) or you can set up a fresh one. Since we&#8217;re using RVM we&#8217;re going to need bash, because RVM is written in bash shell script. One of the maddening things that happened to me was that I was executing the right commands without realizing there was no <code>PATH</code> and so RVM couldn&#8217;t work because bash couldn&#8217;t be found. So if nothing else, create an environment variable named <code>PATH</code> with some value along the lines of:</p>

<pre><code>/bin:/usr/bin:/usr/local/bin:/usr/local/www/.rvm/bin
</code></pre>

<p>You should change that last bit there to the location of the RVM installation you&#8217;re using. Other pertinent environment variables might be things such as <code>RAILS_ENV</code> and <code>RACK_ENV</code> (for &#8220;production&#8221;, &#8220;development&#8221;, etc).</p>

<p>The starting point and probably most intensive part of this is <a href="http://rvm.beginrescueend.com/">RVM</a>. RVM is a set of shell scripts and environment trickery that allows you to install and manage multiple versions of ruby. In addition to this it allows you to separate gems into &#8220;gemsets&#8221; which are unique to each version of ruby you have managed. The gemsets allow you to install a fresh set of gems for each gemset. This was why I was determined to use RVM with my deployment. I don&#8217;t really need multiple versions of ruby itself, but doing that would work just fine with how I did things as well. I installed a limited set of gems that appear in all gemsets (the &#8220;global&#8221; gemset) such as <code>rake</code> and <code>bundler</code> and <code>thin</code> and <code>unicorn</code>. After that, each app has its own gemset with its own local version of rails, rack, and whatever else it needs. This keeps things nicely partitioned and even acts as a sandbox to some degree (to really get sandboxing, <code>bundler</code> offers just superb possibilities).</p>

<p>However, RVM also proved to be the most difficult part of this setup to get working properly. When you&#8217;re working with RVM in your shell you have environment variables and your shell loads RVM&#8217;s magic into itself when you first start the shell. Of course, cherokee isn&#8217;t going to execute your <code>~/.profile</code> when it runs the interpreter for requests. So how do you get RVM&#8217;s magic in there? RVM provides a set of &#8220;wrappers&#8221; that set up all of its environment and magic. These are in <code>~/.rvm/wrappers/</code> by default. So in cherokee&#8217;s information source you need to set up <code>PATH</code> as described above. After that, it&#8217;s really just a matter of perfecting your spawn command. I use unicorn, which only requires a single information source, and my RVM is installed in <code>/usr/local/www/.rvm/</code>, so my spawn command looks something like:</p>

<pre><code>/usr/local/www/.rvm/wrappers/ruby-1.9.2-p180@NameOfApp/ruby /usr/local/www/.rvm/gems/ruby-1.9.2-p180@global/bin/unicorn_rails -c /usr/local/www/NameOfApp/unicorn.conf -E production
</code></pre>

<p>Wow, that&#8217;s a mess. Unfortunately they&#8217;re probably all going to be pretty messy like this. So let&#8217;s break that down:</p>

<pre><code>/path/to/rvm/wrapper/ruby-version@gemset/ruby /path/to/backend --args
</code></pre>

<p>The first command is your ruby executable. Since cherokee doesn&#8217;t know about RVM, in order to allow RVM to set up its magic you have to execute a bash script in the <code>wrappers/</code> directory. This script sets up the necessary magic for the gemset you selected (in this case, <code>NameOfApp</code>) and executes the ruby binary with those gems. The second part is the back end you&#8217;re using. In my case, I execute <code>unicorn_rails</code> which I&#8217;ve installed as a gem in the <code>global</code> gemset (which appears in all gemsets for that ruby version) and pass it a configuration file that tells it what app it&#8217;s serving and how to serve it. Let&#8217;s say you&#8217;re using the default rails server:</p>

<pre><code>/path/to/rvm/wrapper/ruby-version@gemset/ruby /path/to/rails_app/script/server -p port
</code></pre>

<p>This is a little easier, right? In rails-2.3.x (use <code>rails server</code> or <code>script/rails server</code> for rails-3) this would probably launch WEBrick listening on the port that your cherokee information source is expecting (cherokee asks us not to pass any flags to make the server daemonize as cherokee takes care to route the input/output/error streams to useful places and manages other aspects that forking would break). The bit that makes others installed as a gem ugly looking is that you have to find the binary they installed and you have to figure out how to tell them &#8220;go in /path/to/rails_app and run that app in there.&#8221; With unicorn, you can tell it that (and much more) with a unicorn.conf (that doesn&#8217;t need to be in the same directory as the app). With thin, you can tell it that with a configuration file that thin can output using <code>thin config</code> or you can simply run <code>thin start -c /path/to/rails_app</code>. While mine in particular is kind of ugly, it isn&#8217;t as complex as it appears.</p>

<p>Once you figure out the right spawning command, and the RVM scripts can figure out everything they need to run from <code>PATH</code>, you should be good to go. Depending on what back end you&#8217;re using, the first time you try to visit the app&#8217;s address you might get a 502 error from cherokee while it launches the spawning process (typically happens with thin but not with unicorn, and sometimes with <code>script/server</code>) so if you get a 502 try refreshing the page immediately.</p>

<h3>Troubleshooting</h3>

<p>A lot can go wrong. I think probably just about all of it went wrong for me at some point.</p>

<h4>502 Bad Gateway</h4>

<p>This is the error cherokee tosses out if it can&#8217;t get one of your information sources on the phone. This can be because they&#8217;re all busy, but more likely if you&#8217;re reading this it&#8217;s because your spawn command is broken. The best way to investigate this is to set up an error log in the virtual server using the information source that isn&#8217;t working. Cherokee will log the output from the spawn command here. If you see something like <code>ruby not found</code> it&#8217;s because you&#8217;ve gotten the first part of your spawn command wrong. First off, use full paths. Don&#8217;t assume <code>ruby-1.9.2-p180@NameOfApp</code> is in the PATH you set. Sometimes even if it is, it still won&#8217;t run right without a full path. If you see something like <code>env: bash not found</code> it&#8217;s because something or another wasn&#8217;t found in your <code>PATH</code>. Make sure the <code>PATH</code> you set includes anything ruby, RVM, or your back end needs to run.</p>

<p>Another chance is that your spawn command is right but the app you&#8217;re trying to run is having an error. Maybe you&#8217;re missing a gem, or there&#8217;s a syntax error, or any other ruby-level error that&#8217;d cause a backtrace in ruby. You can examine your back end or app logs to find this. Another good way to make sure that your app runs fine is running an equivalent of the spawn command in your shell. Try <code>thin start</code> in an app you&#8217;re running with thin. If it&#8217;s a ruby backtrace you&#8217;ll see it here. Same goes for <code>unicorn_rails</code> or <code>script/server</code>. Generally, if your app runs in your development environment and it doesn&#8217;t run here, it&#8217;s because you forgot something like a gem or other dependency. Also make sure you&#8217;re using the right ruby/gemset in RVM. Running the back end by hand before allowing cherokee to spawn it automatically is always a good idea.</p>

<h4>504 Gateway Timeout</h4>

<p>This usually happens because, while cherokee can get one of your information sources on the phone, it doesn&#8217;t answer in a timely fashion. This can be because you&#8217;re getting slammed and they&#8217;re just running slowly, or more likely because one of the back end processes is attempting to process a request that&#8217;s taking a while. This can be anything from something CPU-intensive to something database-intensive. Examine your SQL queries. check your back end or app logs. For me, this is most often a network or database issue.</p>

<h3>The End</h3>

<p>I hope this helps. If not, and you&#8217;re using cherokee, RVM, and rack apps, my contact information is all over this blog. Feel free to get in touch.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.ericw.org/2011/02/rack-cherokee-rvm/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Why Comcast is Lying</title>
		<link>http://blog.ericw.org/2010/12/why-comcast-is-lying/</link>
		<comments>http://blog.ericw.org/2010/12/why-comcast-is-lying/#comments</comments>
		<pubDate>Thu, 09 Dec 2010 02:27:44 +0000</pubDate>
		<dc:creator>Eric Will</dc:creator>
				<category><![CDATA[technology]]></category>
		<category><![CDATA[comcast]]></category>
		<category><![CDATA[internet]]></category>
		<category><![CDATA[net neutrality]]></category>
		<category><![CDATA[technical]]></category>

		<guid isPermaLink="false">http://pain.ericw.org/?p=122</guid>
		<description><![CDATA[A brief essay on how Comcast is breaking all the rules in order get paid twice for the same thing.</p]]></description>
			<content:encoded><![CDATA[<p>There&#8217;s currently a big debate going on about Comcast, Netflix, and Level 3. Level 3 is saying that Comcast is forcing them to pay them money that really serves Comcast, and Comcast is saying this is a simple peer dispute and not only is Level 3 lying, but Comcast has &#8220;bent over backwards&#8221; to accommodate Level 3.</p>

<p>Comcast&#8217;s argument goes along the lines of a peering dispute. Level 3 is sending much more traffic to Comcast&#8217;s network than Comcast is sending to Level 3&#8242;s network, thus Level 3 must pay (let&#8217;s ignore that this is Comcast&#8217;s very business model: their customers are allotted much more download bandwidth than upload bandwidth, but Comcast feigns surprise). This happened just a few years ago when Cogent was passing much more traffic onto Level 3&#8242;s network than Level 3 was onto Cogent&#8217;s network. Cogent refused to pay up, and Level 3 terminated their peering arrangement. The case of Level 3 vs Cogent was right, and the case of Comcast vs Level 3 is wrong, and here&#8217;s why.</p>

<h3>The Simple Explanation</h3>

<p>Comcast is wrong, because Level 3 isn&#8217;t sending traffic over Comcast&#8217;s network in order to reach a destination on the other side of Comcast&#8217;s network (called <strong>transmit</strong> traffic). Level 3 is delivering Netflix content from its network directly to Comcast customers, because Comcast customers have asked for this content (&#8220;I want to watch this movie, stream it to my face please&#8221;). Comcast&#8217;s subscribers are paying Comcast for this. What Comcast is doing is turning around and making Level 3 pay for the same thing again. They are double-dipping, just as they have in the past. When it was Level 3 vs Cogent, Cogent was <strong>transmitting</strong> traffic <em>over</em> Level 3&#8242;s network to another end point. They were using Level 3 to cut their own costs because, at the time, Level 3 was directly peering with Cogent&#8217;s end destination, and it was saving Cogent money while costing Level 3 money. This is not the same thing. Level 3 is <strong>delivering</strong> traffic directly to Comcast, not using Comcast&#8217;s network to <strong>transmit</strong> traffic to a different end point. Comcast&#8217;s customers are paying Comcast to deliver this content to them, and the people that are doing the actual delivering shouldn&#8217;t only <em>not</em> have to pay Comcast, they should actually be <em>getting paid by</em> Comcast. You pay the UPS driver, not the other way around.</p>

<h3>The Not-As-Simple Explanation</h3>

<p>It&#8217;s a little more complicated than that.</p>

<p>The way the Internet works is that you buy access from your ISP (e.g. Comcast), they in turn (perhaps through some number of additional intermediaries) <strong>buy</strong> access to the Internet&#8217;s core backbone from a Tier 1 provider (like Level 3). If you were a big enough network, you could &#8220;peer&#8221; without paying anything (since there was no particular reason for one network to pay the other or vice versa so long as they exchange roughly the same amount of traffic in both directions). The way it worked in the Old Days was that content providers (like Netflix) would always be uploading more than they download, so the rule of thumb became that if you upload too much data without downloading, you can&#8217;t peer for free. This rule of thumb existed to distinguish content providers from ISPs and make them pay. It is <strong>unheard of</strong> for a backbone provider to pay an ISP. ISPs like Comcast always used to have approximately symmetrical load to their uplink providers: their customers would download more than they upload, which means they would have &#8220;surplus&#8221; upload bandwidth &#8220;for free&#8221; which they could sell to local content providers. Selling to content providers like this is efficient because otherwise the surplus upload bandwidth is lost (&#8220;use it or lose it&#8221;).</p>

<p>Some of these content providers in modern times turned into content delivery networks (CDNs) like Akamai. The CDNs kind of messed with the model. Instead of buying Comcast&#8217;s surplus upload bandwidth to the backbone, they connected to Comcast&#8217;s network directly to serve content only to Comcast&#8217;s customers. In other words, they evened out Comcast&#8217;s uplink to Level 3 not by increasing upload traffic as was the traditional model, but by decreasing download traffic from the backbone. This still works.</p>

<p>So now what&#8217;s happening with Level 3? Netflix is moving from Akamai to a data center operated by Level 3 which is only connected to Comcast through Level 3&#8242;s backbone. What this does from Comcast&#8217;s perspective is both deprive them of the money Akamai was paying them and give them a huge amount of &#8220;free&#8221; upload bandwidth. What Comcast is then supposed to do is shop this surplus bandwidth around to content providers; or, sell Comcast Business customers higher upload rates (or just about anything else) They&#8217;re supposed to sell it, because they can sell it most efficiently; there is a pipe going into Comcast which is only full in one direction and the only way to fill it is to put something which uploads a lot on the Comcast side.</p>

<p>Comcast apparently doesn&#8217;t want to do that. I can really only think of two possible reasons for this. First, there is so much upload bandwidth that no one would possibly want to buy it. I find this to be pretty unfathomable. Supply and demand says that if the price is right, someone will pay. Second, Comcast is double-dipping and trying to sell access to their customers as a scarce resource; in other words they&#8217;re trying to force Level 3 to pay more for the bandwidth than it would sell for on the open market, because buying it from Comcast is the only way to get access to Comcast&#8217;s customers. Comcast has a monopoly on its customers (who already pay them, mind you).</p>

<p>As for Cogent, they were sending traffic over Level 3&#8242;s network to third party networks. That is, they weren&#8217;t delivering traffic to Level 3 customers, they were delivering content to people Level 3 isn&#8217;t getting paid by, while not paying Level 3 themselves. So Cogent was <strong>making</strong> money, and Level 3 was <strong>losing</strong> money. In Comcast&#8217;s case, Level 3 is delivering content to Comcast customers, who <strong>are</strong> paying Comcast for that content.</p>

<p>Level 3 has no monopoly on access to third party networks; there are other backbone providers. Comcast has a monopoly over access to Comcast customers. This is the main difference.</p>

<h3>Why You Should Care</h3>

<p>The people that are calling this a net neutrality issue aren&#8217;t far off: Comcast isn&#8217;t filtering based on content; it&#8217;s actually much worse than that: Comcast is <em>charging content providers</em> in order to allow them to send their content to Comcast&#8217;s customers. This is absurd. If this is allowed to continue (which all of the ISPs are hoping for judging by Time Warner&#8217;s recent endorsement of the Comcast/Level 3 deal) then you&#8217;ll only be able to get Netflix from Sprint, and you&#8217;ll only be able to get Hulu on Comcast, and you&#8217;ll only be able to get to YouTube from, well, whomever YouTube decides to pay. Because ISPs are so large they have the only pipe leading to tens of millions of people and they&#8217;ve decided they should be able to charge the rest of the world to send those people the content those people want to see, which is in fact the very reason those people pay those ISPs to begin with.</p>

<h3>So Why Pay Comcast?</h3>

<p>Level 3 is alleging they agreed to the deal under duress, and are complaining about it now because now is the only time they reasonably could. If they had said &#8220;suck it&#8221; and disconnected from Comcast, any traffic that Level 3 needs to send to Comcast customers (namely Netflix, which they have a contracted obligation to do) would have to be <em>transmitted</em> over one of Level 3&#8242;s peers, and then routed to Comcast.</p>

<p>Ignore that Comcast would march over to the new peer and demand the same thing they&#8217;ve demanded from Level 3. Ignore that Level 3 would undoubtedly have to pay quite a bit of money to this peer to <em>transmit</em> their Netflix traffic to Comcast&#8217;s customers. Doing this would seriously degrade the quality of the video Level 3 is transmitting because now it has to take (at minimum) one more step between Level 3 and the customers that are paying both Comcast <strong>and</strong> Netflix for high quality video. You can bet that Comcast knew they couldn&#8217;t say &#8220;blow us&#8221; without violating their contract with Netflix.</p>

<h3>Not-A-Car Analogy</h3>

<p>Say you rent a DSL line from the telephone company, but (because you&#8217;re currently free to do so) you use an ISP that is not the telephone company. You&#8217;re assigned a bandwidth of 1000Kbps downstream and 256Kbps upstream. You download 200GiB per month, and you upload 50GiB per month from the ISP (which doesn&#8217;t own your line). In this case, your ISP is sending your line far more traffic than you&#8217;re sending back to their lines. Under Comcast&#8217;s new logic, the ISP should be paying you in this situation. You own the line, it costs you money for them to send data to you (never mind you&#8217;ve requested it, just like Comcast&#8217;s customers), and it&#8217;s not balanced because you&#8217;re not costing them the same amount.</p>

<p>Does this make any sense at all to any sane person? Of course it doesn&#8217;t. The ISP would laugh your invoice all the way to the trash.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.ericw.org/2010/12/why-comcast-is-lying/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>New Hope for Novel Analgesics</title>
		<link>http://blog.ericw.org/2010/09/new-hope-for-novel-analgesics/</link>
		<comments>http://blog.ericw.org/2010/09/new-hope-for-novel-analgesics/#comments</comments>
		<pubDate>Fri, 03 Sep 2010 21:38:44 +0000</pubDate>
		<dc:creator>Eric Will</dc:creator>
				<category><![CDATA[medication]]></category>
		<category><![CDATA[pharmacology]]></category>
		<category><![CDATA[research]]></category>
		<category><![CDATA[technical]]></category>

		<guid isPermaLink="false">http://pain.ericw.org/?p=92</guid>
		<description><![CDATA[A lengthy piece examining acetaminophen's metabolic path into AM404 and the activation of the endogenous cannabinoid system.</p]]></description>
			<content:encoded><![CDATA[<p>Pretty much everyone knows what acetaminophen is. If you don&#8217;t, acetaminophen is the active ingredient in the brand names Panadol and Tylenol. Acetaminophen is known by different names&#8211;especially outside the United States&#8211;and is most commonly called paracetamol and often abbreviated APAP (from here on out). All of these names come from the chemical name, n-acetyl-para-aminophenol. APAP is notable as one of the first non-opioid (non-narcotic) analgesics without anti-inflammatory properties (this honor actually goes to the drug phenacetin, which was widely used but taken off the market in 1983 due to carcinogenicity concerns. APAP is a metabolite of phenacetin). It is a pain-relieving (analgesic), fever-reducing (antipyretic) drug in the aniline class, of which itself is the only remaining member.</p>

<p>Until recently, pharmacologists did not fully understand APAP&#8217;s mechanism. That is, exactly how does it relieve pain and reduce fevers? Considering the only other non-opioid analgesics consist entirely of the non-steriodal anti-inflammatory drugs (NSAIDs) such as ibuprofen and naproxen, APAP&#8217;s mechanism was assumed to be a similar one. NSAIDs work by inhibiting enzymes called cyclooxygenase (COX) which produce chemical messengers called prostiglandins which set off inflammation and pain. While NSAIDs markedly reduce inflammation, there is almost little to no inflammation reduction with APAP usage. Why is this?</p>

<p>There are two varieties of cyclooxygenase: COX-1 and COX-2. Most NSAIDs inhibit both of these types equally. COX-1 inhibition has the unwanted side-effect of reducing protective liners in the stomach which can lead to gastric bleeding (indeed, the number one problem with NSAID use). However, inhibition of COX-2 does not produce this effect. Due to this, a number of drugs were developed that selectively inhibit COX-2 while leaving COX-1 alone, and these drugs were called &#8220;COX-2 inhibitors&#8221; with drug name suffixes of &#8220;coxib,&#8221; for &#8220;COX inhibitor.&#8221; Examples of such drugs include valdecoxib, rofecoxib, and celecoxib. A number of these drugs were developed and were very well-regarded by pain management physicians and chronic pain patients alike for their excellent ability to lower pain and inflammation without marked side-effects and even alleviated the need for opioid use (or at least reduced it). Unfortunately some of these drugs were abruptly removed from the United States market and, aside from celecoxib, no new COX-2 inhibitors have been approved or remain on the US market.</p>

<p>So where am I going with this? As APAP&#8217;s mechanism becomes more clear, recent findings have suggested that APAP is strongly selective of COX-2 (so much for the need to remove them from the market). So while APAP does indeed inhibit COX like the NSAIDs, there is strong evidence that APAP works through at least two pathways. The first one is well-researched and well-understood (COX inhibition), and the second pathway is what we&#8217;re interested in. So what exactly is going on here?</p>

<p>Recent research suggests that APAP may earn its analgesic and antipyretic properties by indirectly activating the endogenous cannabinoid system. The same way that opioids activate our own natural pain-relief system that endorphins and other natural ligands use, the body also has a natural cannaboinoid system which is responsible for the effects of tetrahydrocannabinol, or THC, which is the main active ingredient found in marijuana. Just like morphine binds to opioid receptors (mu, kappa, delta, and others), drugs like marijuana bind to the cannabinoid receptors <strong>CB1</strong> and <strong>CB2</strong>. A well-known natural opioid is endorphin. There are also natural cannabinoids, and the one floating around our brains is called <strong>anandamide</strong>. The entire purpose of the endogenous cannabinoid system has yet to be fully elucidated, but we will explore some of the regulatory functions they serve below.</p>

<p>When you take APAP, it is metabolized by the body into a number of different chemicals. Some are active, some are inactive. One particular metabolite is taken in by an enzyme in the body called <strong>fatty acid amide hydrolase</strong> (or FAAH), which converts it into a metabolite called <strong>AM404</strong>. <a href="http://en.wikipedia.org/wiki/AM404">AM404</a> is versatile. It&#8217;s effect is as an analgesic and an antipyretic (sound familiar?). AM404 inhibits FAAH, which also metabolizes <a href="http://en.wikipedia.org/wiki/Anandamide">anandamide</a> (the natural cannabinoid). The net effect is that anandamide uptake is inhibited, and levels of anandamide in the brain increase. AM404 also directly inhibits the formation of COX-1, COX-2, and prostaglandins (sound even more familiar?). AM404 also activates a receptor called <strong>TRPV1</strong>, which is also where the substance capsaicin (the substance that makes hot peppers hot) binds. <a href="http://en.wikipedia.org/wiki/Vanilloid_receptors">TRPV1</a> is responsible for pain transmission and thermoregulatory actions. When activated, TRPV1 enhances and modulates pain transmission, and also tells the body to cool itself down. However, when TRPV1 is bound to for long periods of time it &#8220;shuts down,&#8221; preventing it from functioning, thus reducing pain.</p>

<p>So let&#8217;s take a step back. We&#8217;ve got a lot of things going on. Thanks to AM404&#8211;which is introduced by acetaminophen&#8211;we have the following things going on:</p>

<ol>
<li>AM404 inhibits FAAH&#8211;which <a href="http://en.wikipedia.org/wiki/Anandamide#Synthesis_and_degradation">metabolizes anandamide</a>&#8211;resulting in an increase of anadamide.</li>
<li>Anadamide binds to cannabinoid CB1 and CB2, and also activates the TRPV1 receptor. Each of these actions are known to inhibit pain on their own.</li>
<li>AM404 also activates the TRPV1 receptor.</li>
<li>AM404 also inhibits cyclooxygenase and prostagladins.</li>
</ol>

<p>All of these processes are working to reduce pain (and fever). So, what does this really matter? By investigating these processes we can create novel analgesic drugs that aim to inhibit FAAH in the same way AM404 does (APAP&#8217;s use itself is limited due to its toxicity at higher doses) and giving rise to this exact process. We can also make drugs to target TRPV1, and in fact there are <a href="http://en.wikipedia.org/wiki/Vanilloid_receptors#Clinical_significance">already several</a> in advanced testing phases (both agonists and antagonists are being explored, but I&#8217;d personally be interested in a partial agonist&#8211;can we activate and overload it without causing the burning sensations?).</p>

<p>Let&#8217;s remember, this started by looking closely at the metabolism and mechanism of a drug almost everyone worldwide knows of and has made use of: acetaminophen. First we found out that APAP is most likely a highly selective COX-2 inhibitor, and so that trash about taking Bextra and Vioxx off the market was just that: trash. More importantly&#8211;if you&#8217;ve managed to follow along&#8211;you&#8217;ve almost certainly deduced that because acetaminophen introduces AM404, and AM404 causes activations in the endocannabinoid system, and in this fashion acetaminophen acts as a pro-drug for a cannabimimetic metabolite (AM404 itself), this means that Tylenol and Panadol and hugely popular drugs containing acetaminophen are activating the endocannabinoid system&#8211;like marijuana&#8211;in order to produce it&#8217;s primary effect of analgesia. Tylenol&#8217;s pain-relieving action involves activation of the endogenous cannabinoid system.</p>

<p>And marijuana is illegal?</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.ericw.org/2010/09/new-hope-for-novel-analgesics/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>The Side Effects of Pain</title>
		<link>http://blog.ericw.org/2010/05/side-effects-of-pain/</link>
		<comments>http://blog.ericw.org/2010/05/side-effects-of-pain/#comments</comments>
		<pubDate>Wed, 19 May 2010 18:28:58 +0000</pubDate>
		<dc:creator>Eric Will</dc:creator>
				<category><![CDATA[life]]></category>
		<category><![CDATA[medication]]></category>
		<category><![CDATA[acceptance]]></category>
		<category><![CDATA[adverse effects]]></category>
		<category><![CDATA[coping]]></category>
		<category><![CDATA[prejudice]]></category>
		<category><![CDATA[technical]]></category>

		<guid isPermaLink="false">http://pain.ericw.org/?p=76</guid>
		<description><![CDATA[A semi-long essay about medication side effects (physical and psychological), bias, and other misgivings for people in pain.</p]]></description>
			<content:encoded><![CDATA[<p>I&#8217;ve <a href="http://pain.ericw.org/2010/03/full-circle/">written</a> <a href="http://pain.ericw.org/2010/03/being-a-minority/">previously</a> on the many side effects of having chronic pain. You have physical and psychological side effects from the medications themselves, and physical and psychological side effects just from the pain itself. The former link is about the physical medication side effects, and the latter is about the psychological pain side effects. This piece is going to concentrate on yet more physical medication side effects, but just for a moment I&#8217;d like to expand on the psychological effects from just being in pain.</p>

<p>According to various medical texts, the brain is not designed to be in pain for a long period of time. In fact, the nervous system in general is not designed to send &#8220;pain signals&#8221; for very long at all. As such, when chronic pain happens often times the neurons that send these signals &#8220;learn&#8221; to get better at sending them, in the way that your brain learns for memories. A common myth is that people exposed to pain frequently have a higher &#8220;pain tolerance,&#8221; when in fact the opposite is true. The more often you&#8217;re exposed to pain, the more adept your nervous system becomes at sending pain signals. If the same neurons send the same signals for long enough, something called neuronal plasticity happens: in laymen&#8217;s terms, those neurons physically change themselves to permanently transmit pain. There is no definitive way to tell when something like this has happened to someone, but it&#8217;s a good bet that when all interventional procedures such as disconnecting the nerves themselves (rhizotomy) still fail to provide pain relief, you have this situation. This is bad. There is no way to reverse this process. Your body is now in constant pain. In medical terminology:</p>

<blockquote>
  <p>Under persistent activation nociceptive transmission to the dorsal horn may induce a wind up phenomenon. This induces pathological changes that lower the threshold for pain signals to be transmitted. In addition it may generate nonnociceptive nerve fibers to respond to pain signals. Nonnociceptive nerve fibers may also be able to generate and transmit pain signals. In chronic pain this process is difficult to reverse or eradicate once established.</p>
</blockquote>

<p>Being in constant pain is not only&#8211;you know, painful&#8211;but it also wreaks havoc on your body. People with high-intensity chronic pain have significantly reduced ability to perform attention-demanding tasks. Pain appears to strongly capture the attention of people with chronic pain; tests assessing the ability to attend show poorer performance than pain-free people on all tests demanding attention. The exception is found with tasks that are highly demanding of attention, where performance between the two groups is equivalent. In experimental testing, two-thirds of individuals with chronic pain demonstrate clinically significant impairment of attention independent of age, education, medication and sleep disruption. Individuals with the highest levels of pain showed greatest disruption of memory traces, suggesting that pain diminishes working memory.</p>

<p>Now that I&#8217;ve gotten some of that out of the way, I have another story about physical medication side effects. Last week I was at work. I work some two hours from my home. Right before leaving, I went to use the bathroom but found it difficult to urinate. I didn&#8217;t really give this much thought because I&#8217;ve had a &#8220;<a href="http://en.wikipedia.org/wiki/Paruresis">shy bladder</a>&#8221; for close to a decade. It&#8217;s always been stressful and difficult for me to give urine samples for job applications or other reasons. I left work and when I got home a few hours later, I was still unable to urinate. This worried me. Since I usually get home late I tried a few more times and went to bed. When I got up in the morning and got to work I found myself still unable to urinate. At this point I was still leaning toward a shy bladder and figured that if I drank enough I could sort of force it out. This is the technique I use to give urine samples. A few hours later I was very much surprised to find that this didn&#8217;t work.</p>

<p>After consuming a large amount of fluids I felt as if my bladder were going to burst and I was in quite a large amount of pain and discomfort. I immediately ran to the bathroom where I was still totally <a href="http://en.wikipedia.org/wiki/Urinary_retention">unable to urinate</a>. I realized at this point I had an emergency condition on my hands and tried to convince my carpool to start heading home immediately. By the time they had gathered their things I realized there was no way I was going to make it two hours back home, and decided to run to the hospital across the street from work.</p>

<p>Upon walking into the ER I told the triage nurse I hadn&#8217;t urinated for 22 hours and I was in severe discomfort. After a quick sign in with a list of medications and allergies I was rushed off to a room where she quickly (and not as horrifyingly painfully as <a href="http://pain.ericw.org/2010/03/full-circle/">last time</a>) inserted a Foley catheter where she promptly drained close to 950mL of urine (a normal liquor bottle is 750mL). I felt much better. After a discussion with the physician about the possible causes he was leaning toward my pain medication, as opioids are a frequent cause (but individually rare (less than 10%) side effect) of urinary retention. After a urine analysis (I didn&#8217;t have trouble giving a sample this time&#8211;it came out of a tube), blood work, and a couple of non-contrast abdominal and pelvic CT scans all came back normal I was told the most likely culprit was the &#8220;vast amount of narcotic medications&#8221; and advised it would be best the catheter stayed in a few days. I was not entirely happy with that verdict. As I limped out of the ER (and when you&#8217;re a male with a tube up your penis, you <em>limp</em>) I was picked up by my carpool and taken home. After a few days of being told my PCP wouldn&#8217;t take out a catheter in the office and to go to the ER I realized I could either spend five hours at the bottom of the ER triage or I could take it out myself. I cut the injection port and let the balloon deflate and gently removed it without issue. Within a few hours (and since) I have been urinating just fine.</p>

<p>This story is obviously personal and represents a few things. The physician determined my situation was a side effect of my pain medication and so informed me in such a way as to say &#8220;maybe you shouldn&#8217;t be taking those medications.&#8221; Well, doc, I&#8217;d love to not take these medications but without them I can&#8217;t function due to <a href="http://www.intractablepaindisease.com/">Intractable Pain</a>. Maybe if I was on Disability or some other form of I-don&#8217;t-work income I could try it and see what happens, but I have a job and a family to support with that job. So firstly it represents bias against people on pain medications, once again. This is a psychological (or maybe even social) side effect of pain medications. Friends and family may shy away from you because you&#8217;re on &#8220;narcotics&#8221; or worse, assume you&#8217;re an addict because you&#8217;re dependent. I&#8217;ve <a href="http://pain.ericw.org/2009/01/neuropharmacology/">written extensively</a> on how these differ, but the laypeople just fail to understand. There have even been episodes of <em>Intervention</em>-style shows on television wherein a chronic pain patient was accused by a weeping family of abusing their medications and being an addict. A lot of times on these shows it&#8217;s true: they&#8217;re quite obviously addicts; however, more than once I&#8217;ve seen a pain patient taking their medications as prescribed and enduring their side effects only to have this taken as &#8220;addiction.&#8221; What&#8217;s worse is that I&#8217;ve seen these legitimate patients forced into rehab more than once because of a bias against medication.</p>

<p>We are sick people. We are sick people with a serious disease. We have pain that&#8217;s caused by a disease or is a disease in and of itself. We take medication for that disease, and we withstand the side effects of both that medication and that disease. My only problem is that some of those side effects are man-made, and in a reasonable and educated world should be put to rest. Educate yourselves.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.ericw.org/2010/05/side-effects-of-pain/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>The Truth About Tolerance</title>
		<link>http://blog.ericw.org/2010/02/the-truth-about-tolerance/</link>
		<comments>http://blog.ericw.org/2010/02/the-truth-about-tolerance/#comments</comments>
		<pubDate>Sat, 06 Feb 2010 00:03:27 +0000</pubDate>
		<dc:creator>Eric Will</dc:creator>
				<category><![CDATA[medication]]></category>
		<category><![CDATA[pharmacology]]></category>
		<category><![CDATA[dependence]]></category>
		<category><![CDATA[research]]></category>
		<category><![CDATA[technical]]></category>
		<category><![CDATA[tolerance]]></category>

		<guid isPermaLink="false">http://pain.ericw.org/?p=43</guid>
		<description><![CDATA[A very medical-ese explanation of NMDA-mediated opioid tolerance.</p]]></description>
			<content:encoded><![CDATA[<p>A long-experienced phenomenon, tolerance is the need to increase the dosage of medication to achieve the same effect. This is most frequently seen in illicit drug abuse. Heroin users quickly have a need to take increasing doses in order to achieve the same &#8220;high.&#8221; The same goes for other illicit drugs like amphetamines and cocaine. With most drugs tolerance is a complicated process and is not fully understood. In most cases we assume the <a href="/2009/01/neuropharmacology/">neurotransmitters</a> (or their receptors) affected by the specific drug type is down-regulated in some way. That is to say, if you take a lot of opioid analgesics your body reduces the amount of naturally-produced (endogenous) opioids, and also decreases the amount of opioid receptors in the body.</p>

<p>Some time ago, it was noticed that NMDA receptor antagonists (dissociative anesthetics) like ketamine, phencyclidine, and dextromethorphan have the side-effect of reducing the amount of tolerance formed to opioid analgesics. This has far-reaching implications because if you can mediate opioid tolerance, you can control the amount of opioid needed for pain relief.</p>

<p>The NMDA receptor both induces and maintains persistent enhancements of the excitability of neurons to prolonged stimulation, or &#8220;wind-up.&#8221; Wind-up is a key spinal mechanism requiring activation of the NMDA receptor that both amplifies and prolongs certain types of pain. As a result, wind-up may be one of the events underlying prolonged or chronic pain. Evidence from <a href="/2010/02/research-relieves/">animal studies</a> indicates that this mechanism is involved in the induction and maintenance of certain types of pain, most notably inflammatory and neuropathic.</p>

<p>Neuropathic pains are at least partly mediated by the NMDA receptor, which may relate to changes in opioid sensitivity. All opioids reduce, or with high doses block the input that causes certain types of pain, probably via activation of the presynaptic opioid receptors to prevent the release of primary afferent transmitters and so prevent pain input from actually activating the neurons that make you feel pain. However, if the pain continues, wind-up overcomes the inhibitions of input and the neurons commence firing, causing pain. As wind-up increases the activity of neurons, a higher dose of opioid will be required to block the increased excitability. Thus, at moderate doses, opioids delay wind-up without inhibiting the process itself. In contrast, NMDA antagonists abolish wind-up. Thus, threshold doses of morphine combined with low doses of NMDA antagonists are able to elicit <em>dramatic</em> inhibitory effects, a synergism that suggests low probability of side effects. Importantly, in a model of neuropathic pain where morphine is inoperative, the co-application of an NMDA antagonist restored the ability of morphine to inhibit the response.</p>

<p>All that medical speak translates to this: the pain input that&#8217;s prolonged and intensified by NMDA receptors can be delayed by opioids, but not inhibited. However, NMDA antagonists (mentioned above) completely turn off the prolongation and intensification, allowing opioids to take away that pain. Basically stated, it amounts to the aforementioned. Adding a mild NMDA receptor antagonist (in extremely sub-anesthetic doses) to an opioid enhances the effects of the opioid, allowing smaller amounts of opioid, and thus fewer side-effects.</p>

<p>At least there&#8217;s one good use for dextromethorphan.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.ericw.org/2010/02/the-truth-about-tolerance/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
	</channel>
</rss>

