Starfish1 bills itself as “a utility to make distributed programming ridiculously easy.” I think that is a bit of an overstatement, but Starfish certainly can make distributed programming easier.
This chapter first looks at how to use Starfish to facilitate communication between distributed programs. Then you’ll find out more about how we can use it to write MapReduce functions.
Including 8 parts:
Installation
Getting Started with Starfish
"Hello World" the Starfish Way
Building a Distributed Logger with Starfish
Letting Starfish Shine
MapReduce and Starfish
Conclusion
Endnotes
Installation
command: gem install starfish
show: Successfully installed starfish-1.2.1
Getting Started with Starfish
The documentation for Starfish gives a simple piece of sample code to get us into the flow of the library
foo.rb
运行命令： starfish foo.rb
然而我在windows上运行报错找不到server和client定义…拿不准是因为系统还是什么，隐约觉得和ruby的那些magic方法有关，毕竟刚接触ruby不久。虽然之前看过说”如果要学ruby on rails千万不要用windows，重要的事情说三遍之类的言论“先这样吧…私以为本章的例子都要跪…
“Hello World” the Starfish Way
We’ll start with writing it using the starfish binary that comes bundled with the gem. Then we’ll look at what we would need to do to get the same application to work without the help of the binary.
Using the Starfish Binary
In the brief example that comes with Starfish, we saw the server and client both written in the same file. Although that is great in a sample file, in real life we would most likely want to have those in two separate files, potentially even running on different machines.
save it as "server.rb" command: starfish server.rb
当然…我还是运行失败了…
Saying Goodbye to the Starfish Binary
嗯…还是失败。这个例子在Page93
Building a Distributed Logger with Starfish
Letting Starfish Shine
MapReduce and Starfish
The MapReduce design paradigm is simple. It maps a collection of data and then reduces it to a smaller collection.
接下来讲了两个例子 Page104
Using Starfish to MapReduce ActiveRecord
Using Starfish to MapReduce a File
Chapter5: Distribunaut
For all of its pros, Rails has a few cons as well. I won’t go into all of them now, but the biggest disadvantage we found was that Rails doesn’t want to help you write complex portal applications. It wants you to build smaller, simpler applications—at least,at the time it did. With Rails 3.0 on the horizon, that may change.
In addition to building this large portal, we decided we wanted to split our application into many applications. The advantages we saw were smaller code bases that were easier to maintain and separate applications that were easier to scale. We also could push updates and new features sooner, because we didn’t have a gigantic code base to worry about.
We identified three main problems. First, we wanted to let each application maintain its own set of routing, but we wanted the other applications to be able to use the dynamic routing we had become accustomed to in Rails. We didn’t want to hardcode URLs in the other applications; we wanted them generated by the application they would be linking to. Second, we wanted to share views and layouts among these applications. We didn’t want to have to deal with SVN externals, GIT submodules, or symlinks. We wanted to be able to quickly say, “Here is a URL for a layout. Render it like you would a local layout.” Finally, we wanted to share models and libraries throughout all these applications without having to worry about packaging them and redeploying all these applications each time we made a bug fix to a model.
Distribunaut2 is a port of one-third of the distributed features that are found in Mack. In particular, it focuses on making it incredibly easy to distribute models and other Ruby classes. You will not find distributed views/layouts and routes in Distribunaut. The reason is that they are too specific to each of the web frameworks out there, and coding for each one would be a lot of work.
Including 7 parts:
Installation
Blastoff: Hello, World!
Building a Distributed Logger with Distribnaut
Avoiding Confusion of Services
Borrowing a Service with Distribunaut
Conclusion
Endnotes
Installation
Blastoff: Hello, World!
好吧…运行 distribunaut_ring_server start 报错
后面有个使用distribunant的例子
Building a Distributed Logger with Distribunaut
依旧是个例子，不运行了
Avoiding Confusion of Services
Earlier, when speaking about application names, I mentioned that names need to be unique to avoid confusion, but I didn’t explain what I meant.
这样主要针对unique application name的问题
Borrowing a Service with Distribunaut
Conclusion
Endnotes
这章好像就列了列标题啊…好吧，略看
Chapter6: Politics
The problem we were running into was this: How do we configure just one machine to run our background jobs, scheduled tasks, and other queue-type processing? We didn’t want each machine to perform these tasks for a couple reasons. First,we didn’t want to worry about each of these processes to step on the other’s toes, processing the same task. Second, we didn’t want to add load to each instance, when we could burden just one instance.
Politics provides modules that allow us to build a self-repairing worker class that can be run on all our instances. But it designates only one instance at a time to do the work specified for a given time period. Politics calls this worker a token worker. Under the covers Politics uses three different technologies to maintain order in all the worker classes. One of them we already know—DRb. The other two technologies it uses are Memcached and mDNS (also known as Multicast DNS, Zero Configuration Networking, or Bonjour).
Including 4 parts:
Installation
Working with Politics
Conclusion
Endnotes
Installation
看来这章的例子也只能看看了…
To use Politics, you must first have both Memcached and mDNS installed on your system. If you are using Mac OS X, you already have mDNS installed. The same goes for most flavors of UNIX.
Memcached will need to be installed on most machines. Instructions on how to do this can be found at http://www.danga.com/memcached/.
As I mentioned earlier, one of the problems I was having was making sure that all the processes I had running didn’t step on each others’ toes and spend cycles doing the same work. This is a common problem when dealing with distributed programming. A typical example occurs when we are working with distributed message queues. Part III of this book, “Distributed Message Queues,” discusses distributed messaging queues in greater detail. For now, simply know that what we discuss in this section can make processing those queues extremely effective and powerful.
What does this module offer us? The Politics::TokenWorker module, when included into a worker class, allows us to create a class that will either process the work we give it and act as the “leader,” or patiently sit and wait for its turn to be the “leader”while not doing any work.
