Requirement: 800+ function inputs/outputs (pinmux)

Support
1

so whilst that may sound completely and utterly insane, to have a python (verilog) function with over 800 inputs and outputs, for a pinmux that’s on the reasonable side.

background: i am currently designing a libre pinmux auto-generator (written in python) that is being designed to aid and assist the process of creating a pin multiplexer for SoCs. documentation will be auto-generated, device-tree files for linux kernels will be auto-generated, header files for inclusion in arduino libraries, linux kernel source code, *BSD kernel source code, libopencm3, everything. all the things that are done by hand right now that have to be done by hand because most of the time a pinmux is implemented by proprietary companies.

the first language-output mode is progressing well, it is for bluespec’s BSV. now with testing underway it is time to move on to myhdl.

and there i have a bit of a problem. the auto-generated output from bluespec’s BSV compiler automatically unpacks any “objects” that are passed to it, such that the wires IN those objects (three in this particular case) end up as auto-generated verilog with over EIGHT HUNDRED arguments.

for a pinmux which has extremely complex inter-dependencies this really is the only way.

the pinmux that i am working with has 160 pins (rows), and 4 muxer entries (columns). so object “1” is the muxer settings (QTY 160 wires @ 2 bits each).

object 2 is the iopads (QTY 160, @ 3 wires each: in-wire, out-wire and direction-setting-wire).

object 3 is the peripherals, of which there are around 30, each with between 2 and 96 “pins”, where each “pin” has, again, depending on function type, an input wire, and output wire, and sometimes both so it also has its own direction-control wire.

bluespec’s BSV compiler handles this really well by allowing objects to be passed as function parameters. that will be three objects: one for the muxer (with a list of 160 wires), one for the iopads (with a list of 480 wires), and one for the peripherals (with hundreds more wires).

question.

does myhdl have the ability to pass objects into functions, where the objects will be recognised as having input and output wires in them, such that i don’t have to have an insane number of arguments to a function?

is it possible to pass in lists of wires for example?

much appreciated.

2

Ideally you want to handle it as:

PinMux(MuxDescription, Peripherals, MuxSelect, IoPins):
    . . .

Currently MyHDL doesn’t handle lists of interfaces at the top level in conversion, but you can work around that using bitvectors Signal(intbv()[N:]) for the top-level function:

PinMux(MuxDescription,
       PeripheralsIn, PeripheralsOut, PeripheralsDir,
       MuxSel
       IoIn, IoOut, IoDir,):
    . . .

In the function assign the bits to objects and proceed from there.

3

hi josyb thanks for responding.

yowser, that’s a bit-vector 160 wide! as long as the underlying simulation type uses Long (not int) that should be viable. i particularly wanted to use the actual python code (executed as python, not a simulation) to test the other HDLs.

ok the top-level assignment idea should do the trick.

one thing that is quite annoying, i need to give all of these bits meaningful names (particularly for the functions and the pins). is it possible to pass dictionaries at the top-level instead of lists (and the dictionaries to contain the names of the variables to go into the verilog top-level signals/wires/regs)?

or, is it possible somehow to over-ride the name of an object? so for example, pass in a list of signals at the top level, where each Signal has had a special function called which says “hey please use this manual name not an auto-generated one”?

4

I have simulated 512-bit AXI4 buses , so no worries there.
If you want to assign specific names for each you will have to spell out every SIgnal at the top-level. Using interfaces will reduce the load somewhat, as these are expanded by the conversion.
The top-level would than group the signals into lists of signals going to/coming from the PinMux function. The MuxDescription will help to automate that.
A sketchy example, e.g muxing 2 I2C and 2 SPI onto a set of 8 pins, would help in testing this all?

@block
def toplevel(Description, I2C0, I2C1, Spi0, Spi1,
             MuxSelect,
             PinIn, PinOut, PinDir):

    PeriherlpheralsIn = []
    PeriherlpheralsOut = []
    PeriherlpheralsDir = []
    for item in MuxDescription:
        if item.isinput:
             PeriherlpheralsIn.append( fetchargument(item.signame))
        elif . . . 
        elif . . .

   pm = PinMux(MuxDescription,
               PeripheralsIn, PeripheralsOut, PeripheralsDir,
               MuxSel
               IoIn, IoOut, IoDir,)

  return pm
5

it’s a good idea… the problem is that the muxer is speciied dynamically,
at run-time, from objects that come either from TSV-formatted files (or
from objects in python). so the only way(s) to create that top-level
function is:

(A) create a template as a string, save that to a file and then import it
(dynamically)
(B) create a template as a string and exec it
© construct and/or otherwise manipulate in-memory python AST objects

basically the user is permitted to specify dynamically what
functions/peripherals they want. the pins at the moment will be given
numerically-sequential names so that’s not a problem, likewise the muxer
settings. those don’t change. the peripherals on the other hand, that’s
really essential. i really don’t want to have to use the techniques
above.

what would work is if the toplevel accepted *args. that would then allow
arbitrary peripherals to be passed in (as a list, effectively).

l.

6

what would work is if the toplevel accepted *args. that would then allow
arbitrary peripherals to be passed in (as a list, effectively).

oh: i had a brainwave. it might be possible to construct a decorator
that hand-grafts a list of arguments to a function. hypothetically it
would be possible to deepcopy the function prior to adding the decorator…
hmm…

l.

7

it’s a good idea… the problem is that the muxer is speciied
dynamically, at run-time, from objects that come either from TSV-formatted
files (or from objects in python). so the only way(s) to create that
top-level function is:

(A) create a template as a string, save that to a file and then import it
(dynamically)

haven’t tried this one yet… really don’t want to.

(B) create a template as a string and exec it

tried this: unfortunately the use of inspect.findsource and getsourcelines
completely fails… as the exec’d object doesn’t actually have a file to
read / be read from. gaah.

(C) construct and/or otherwise manipulate in-memory python AST objects

with B failing i doubt C would work either

damn this is way, way harder than it should be!

l.

8

I would do this: have my Python generate a MyHDL source file and start a child process to run it. But then I’m only a simple HW engineer with only basic SW knowledge, at least I didn’t get any formal education in the SW domain, actually neither in the HW domain …
Building on my sketchy code, things to do is to generate that single first line with all the objects:
def toplevel( MuxDescription, Interface0, interface1, ... ),
paste the static code to assign signals to objects and back
and close the function

then declare all the signal objects
instantiate the module
write the line to convert/simulate, conversion is short, simulation may take work as we need stimuli

Is anything of your project visible on the web?

11

ok success! it’s ugly as hell but it works:
http://git.libre-riscv.org/?p=pinmux.git;a=blob_plain;f=src/myhdl/pins.py;hb=fcfe66696d70a16981c8b6435ef7341b29b220ef

the primary function which is being auto-generated is called test (because this is a test) and it’s created in… um… create_test (duh) :slight_smile:

create_test takes the number of pins N and the number of functions F as parameters. the auto-generated function has a hard-coded number of extra parameters equal to N*2 + F

its job is to take those extra auto-generated parameters and pack them into a single list, then call down to a function which has a set number of parameters.

this function is where the muxer work will take place, and it does not have to have arbitrary numbers of hard-coded parameters.

i checked also that classes can be passed as parameters at that top level: they can, which is great. that’s a lot less complicated than i was anticipating.

12

ok so i have been forced into the position of creating a censorship bypass account, because the censorship that jck has enacted has an unfortunate side-effect, which you may not have been aware of, jck, which is that it terminates all and any communication including sending simple messages to the moderators such as “why on earth is this discussion being treated as spam???”

so the only way to reach the moderators is to use public discussion.

can you please contact me by email - luke.leighton@gmail.com - with an explanation?

josyb specifically requested links be provided to the online (libre-licensed) code, so that he could take a look at it. it cannot possibly be classed as “advertising”. this is a complex area of programming and i need help. in turn, people will learn from that help, and, furthermore, i will then be able to begin improving myhdl.

but if you start censoring discussions how the hell am i supposed to help anyone??

much appreciated your prompt explanation.

13

Quite unfortunate indeed!
Are you sure it was @jck who blocked you?

14

censored
censored744×264 15.5 KB

yes. i’m now blocked from replying to threads, i’m blocked from sending enquiries to the moderators, i’m blocked from creating new threads, everything.

15

testing, testing… ok! and it’s back. (thanks for sorting that out jck).

16

@jck is the super-moderator who set up the discourse, so he gets the blame even if it was an automated action by the spam-filter, which evidently works well, as we didn’t see any spam yet. Your message must have been that 0.0000005 point above the threshold :slight_smile: Maybe it was the large link?

I was looking for a sort of description on an actual (test) pinmux description but couldn’t find much useful. I got the bluespec generation working on my system after editing the imports but that didn’t learn me much. I started my own small project, but did’t get round to finish it. Now probably never will.

17

ahh ok so an automated system kicked in under his name sigh got it.

ok so there’s not very much documentation on this code, as it’s really quite recent. there are two aspects to the program: specifications, and code-generators. specifications are listed in src/spec/init.py, and generators are in src/bsv, src/myhdl and so on, only one of which is done at the moment.

so the bluespec stuff was the first back-end, working with IIT Madras we just got to the point where we wrote the first unit tests of a small pinmux. that unit test (src/test_bsv/tests/test_pinmux.py) was written by hand, and hard-coded to one specific muxer (src/spec/microtest.py). it’s a cocotb test, and it needs a “model” to compare what it’s simulation.

so obviously the next phase is to auto-generate the unit tests from the specifications, so that the unit tests do full coverage. that’s where, only a few days ago, i realised that if the unit test models were to be written in python, i might as well write them in myhdl.

you shouldn’t have needed to edit the imports, as the auto-generator actually generates the Makefile for the spec, as one of the output files. if it didn’t work do let me know! these commands should have generated microtest/bsv_src/Makefile etc. which you can just use:

python src/pinmux_generator.py -s microtest -o microtest python src/pinmux_generator.py -o microtest

you should also see a markdown file auto-generated, microtest/microtest.mdwn which is the beginnings of a technical reference manual for the specification (src/spec/microtest.py)

do you mean, you started writing a pinmux at some point? because if you’d like to help with this one that would be real handy :slight_smile: i have the base classes / infrastructure there, which reads stuff. it’s ironic that an actual muxer really is tiny, one of the very first verilog tutorials i ever did, but like a DRAM cell != DDR3, a mux cell != a pinmux generator!

18

ok i just added in some code that parses the intermediate TSV-separated files (e.g. those in microtest after running python src/pinmux_generator.py -s microtest -o microtest) and then separated out a base class which reads them in and generates some (stub-ish) classes named “Pins”.

http://git.libre-riscv.org/?p=pinmux.git;a=blob;f=src/myhdl/pinmux_generator.py

the “Pins” class in that file I will replace with Signal() pretty much one-for-one. or perhaps i will replace it with the IO class here:

http://git.libre-riscv.org/?p=pinmux.git;a=blob;f=src/myhdl/pins.py

the next phase would be to create a “proper” version of that create_test function, which takes the Interfaces object (now containing the list of interfaces like twi0, gpioa, sdmmc0, sdmmc1 and so on) and magically make a python module containing a myhdl function, to which the “Pins” (or more likely “IO”) objects can be passed.

the amount of auto-generated code that’s going to be spat out by this is… vast. the BSV auto-generator produces 17 THOUSAND lines of code from a 160x4 pinmux. it’s kinda horrifying and fun at the same time… :slight_smile:

19

I was just hoping to find out more how a pin-mux is specified …

yes, I can only straighten my thoughts on the anvil.
And I have trouble understanding your (and most of the time anybody else’s) code.
I’ll post/send my code as soon as it is somewhat presentable, but don’t hold your breath, I have a demanding day-job …

20

ah. right. ok. so what i did was to write it out as python code, and to have some more code turn that into for example a markdown file, after carrying out some checks.

so this page is entirely auto-generated for example: pinouts

but it’s generated from this: git.libre-soc.org Git - pinmux.git/blob - src/spec/m_class.py

it’s deliberately written to be clean and obvious what’s going on. the bits that aren’t explained / obvious are the “scenarios”. those are basically, “i have a schematic, i want certain functions, i don’t care necessarily which interface to connect to, i just want an answer to my question, is my SCENARIO viable yes or no?”

and what the add_scenario function will do is, lift those pins on a per-function basis out of the available pinmux, and if as a result of doing that one function does not HAVE pins available, an error is reported.

so you can TEST the proposed pinmux as being viable, for multiple scenarios, BEFORE you go and spend the time and money.

the next phase of development will get the code to auto-generate a complete and full Technical Reference Manual, and the #include header files (which are a pain in the ass to write by hand), and i’d really like to see actual GPIO driver source code be auto-generated as well.

you and me both :slight_smile:

no problem. it would be really handy, i’m fighting myhdl here. i have a conference to go to which is in a week (RISC-V 9th workshop in Chennai) so will be a little distracted by that anyway.