扭曲的插件错误

Question

A have created a dead simple twisted application that starts a TCP protocol and echo out what you type in STDIN.

I am now trying to create a twistd plugin to be able to run my application this way: echo start or either twistd -n echo

When running twistd -n echo everything works as expected, when using the echo start command I get the error: /home/vagrant/.env/bld/bin/echo: Unknown command: echo

Here is my code:

echo/plugins.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from twisted.application import internet
from twisted.internet import endpoints
from twisted.internet.protocol import Factory
from twisted.python import usage

from echo.protocol import EchoProtocol


class Options(usage.Options):
    optParameters = [['port', 'p', 1234, 'Service port.']]


def makeService(options):
    from twisted.internet import reactor

    f = Factory()
    f.protocol = EchoProtocol

    ep = endpoints.TCP4ServerEndpoint(reactor, int(options['port']))
    return internet.StreamServerEndpointService(ep, f)

echo/protocol.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from twisted.internet.protocol import Protocol


class EchoProtocol(Protocol):
    def dataReceived(self, data):
        self.transport.write('You entered: {data}'.format(data=data))

echo/tap.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys

from twisted.python import usage
from twisted.scripts import twistd


class Start(twistd.ServerOptions):
    def parseOptions(self, args):
        sys.argv[1:] = self.getArguments(args)
        print('Starting echo service...')
        twistd.run()

    def getArguments(self, args):
        args.extend(['--pidfile', self.parent.pid])
        args.extend(['_bld_echo'])
        return args


class Options(usage.Options):
    pid = '/tmp/echo.pid'
    subCommands = [['start', None, Start, 'Launch echo service.']]


def main(argv=None):
    o = Options()
    try:
        o.parseOptions(argv)
    except usage.UsageError, e:
        raise SystemExit(str(e))

twisted/plugins/echo_plugin.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from twisted.application.service import ServiceMaker

Finger = ServiceMaker(
    'EchoServiceMaker',  # name
    'echo.plugins',  # module
    'Description blah-blah.',  # description
    '_plgn_echo')  # tapname

setup.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from setuptools import find_packages
from setuptools import setup

setup(
    name='Echo',
    version='0.0.1',
    packages=find_packages(),
    entry_points={
        'console_scripts': [
            '_ep_echo=echo.tap:main',
        ],
    },
    install_requires=[
        'Twisted==16.0.0',
    ],
    include_package_data=True,
    zip_safe=False,)

Here my virtualenv setup:

(bld)vagrant@/code/echo $ pip list
Echo (0.0.1)
pip (1.4.1)
setuptools (20.3.1)
Twisted (16.0.0)
wsgiref (0.1.2)
zope.interface (4.1.3)

I have prefixed my commands with _ep_ and _bld_ because I was not sure which one were called when invoking the program through twistd or by directly calling the entry_point, but I have tried any possible combination without success...

When I run _ep_echo start I get:

[twistd -help output...]

twistd reads a twisted.application.service.Application out of a file and runs
it.
Commands:
    conch            A Conch SSH service.
    dns              A domain name server.
    ftp              An FTP server.
    inetd            An inetd(8) replacement.
    mail             An email service
    manhole          An interactive remote debugger service accessible via
                     telnet and ssh and providing syntax coloring and basic line
                     editing functionality.
    manhole-old      An interactive remote debugger service.
    news             A news server.
    portforward      A simple port-forwarder.
    procmon          A process watchdog / supervisor
    socks            A SOCKSv4 proxy service.
    telnet           A simple, telnet-based remote debugging service.
    web              A general-purpose web server which can serve from a
                     filesystem or application resource.
    words            A modern words server
    xmpp-router      An XMPP Router server

/home/vagrant/.env/bld/bin/_ep_echo: Unknown command: _bld_echo

The same goes if I replace _bld_echo with _ep_echo .

One thing is weird when looking at the output: the twistd does not have the echo subcommand registered.

If I run twistd --help I get:

twistd reads a twisted.application.service.Application out of a file and runs
it.
Commands:
    _plgn_echo       Description blah-blah.
    conch            A Conch SSH service.
    dns              A domain name server.
    ftp              An FTP server.
    inetd            An inetd(8) replacement.
    mail             An email service
    manhole          An interactive remote debugger service accessible via
                     telnet and ssh and providing syntax coloring and basic line
                     editing functionality.
    manhole-old      An interactive remote debugger service.
    news             A news server.
    portforward      A simple port-forwarder.
    procmon          A process watchdog / supervisor
    socks            A SOCKSv4 proxy service.
    telnet           A simple, telnet-based remote debugging service.
    web              A general-purpose web server which can serve from a
                     filesystem or application resource.
    words            A modern words server
    xmpp-router      An XMPP Router server

And there you can see the echo command registered.

This is driving me crazy, any ideas about what's the problem here ??

Notice that I run python setup.py install and not python setup.py develop , the latter command works but I do not want to run that in production

---

EDIT

Ok after searching why the axiomatic start worked and not my echo start I found out the reason by removing all unneeded code from the install and here what I have found (I do not claim this is the solution, I would love to hear @glyph answer on this)

The major difference between Axiom and Echo is this line in setup.py :

packages=find_packages() + ['twisted.plugins']

I did not have the + ['twisted.plugins'] addition to the packages line and now it works, but there is still this error happening:

Unexpected error while writing cache file
Traceback (most recent call last):
  File "/home/vagrant/.env/bld/lib/python2.7/site-packages/Twisted-16.0.0-py2.7-linux-x86_64.egg/twisted/application/app.py", line 579, in parseOptions
    usage.Options.parseOptions(self, options)
  File "/home/vagrant/.env/bld/lib/python2.7/site-packages/Twisted-16.0.0-py2.7-linux-x86_64.egg/twisted/python/usage.py", line 262, in parseOptions
    for (cmd, short, parser, doc) in self.subCommands:
  File "/home/vagrant/.env/bld/lib/python2.7/site-packages/Twisted-16.0.0-py2.7-linux-x86_64.egg/twisted/application/app.py", line 596, in subCommands
    for plug in sorted(plugins, key=attrgetter('tapname')):
  File "/home/vagrant/.env/bld/lib/python2.7/site-packages/Twisted-16.0.0-py2.7-linux-x86_64.egg/twisted/plugin.py", line 213, in getPlugins
    allDropins = getCache(package)
--- <exception caught here> ---
  File "/home/vagrant/.env/bld/lib/python2.7/site-packages/Twisted-16.0.0-py2.7-linux-x86_64.egg/twisted/plugin.py", line 185, in getCache
    dropinPath.setContent(pickle.dumps(dropinDotCache))
exceptions.AttributeError: 'ZipPath' object has no attribute 'setContent'

The plugin works but I would really love to know why my original way of installing it did not work...

Answer 1

Thank you for an extremely thorough explanation of your problem; I was able to reproduce it exactly with only one minor tweak (creating a __init__.py inside echo/ to make it a proper package).

First, here's the fix:

diff --git a/echo/tap.py b/echo/tap.py
index d23571f..8e1ea84 100644
--- a/echo/tap.py
+++ b/echo/tap.py
@@ -15,7 +15,7 @@ class Start(twistd.ServerOptions):

     def getArguments(self, args):
         args.extend(['--pidfile', self.parent.pid])
-        args.extend(['_bld_echo'])
+        args.extend(['_plgn_echo'])
         return args

The reason that this works is that what you are doing when you are writing a command like this is you are wrapping the execution of the Twisted plugin , which means the thing that goes on your synthetic command line constructed by args.extend is the tapname of the Twisted plugin , the same thing that would go on the twistd command line.

Hopefully it's clear why this is _plgn_echo .

I also have to commend you on adding these prefixes to clearly try to clearly understand which name, particularly, is being referred to in each area of the code. Assuming that this answer makes sense to you, you will have a far better understanding of the code here than you would have if you had just stuck echo everywhere, even if it had happened to work at first :-).