Calculating Simulation Time Using Bash
Thursday October 01st 2009, 5:26 pm
Filed under:
Script
In our verification job, we often have to care about how long RTL simulation last. That way we can plan verification project and put the resource to meet global project schedule. Here’s an example of how to calculate RTL simulation time using bash. The time jitter is approximately bellow second, but it’s good enough to examine simulation time that usually last more that one minutes, especially in RTL simulation.
Complete scripts:
start=$(date +%s)
time sleep 5 # execute your
# RTL simulation script
# here
finish=$(date +%s)
echo "Simulation time: $((finish - start)) secs"
Real Test:
$ ./test
0.00user 0.00system 0:05.00elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+207minor)pagefaults 0swaps
Simulation time: 5 secs
Enjoy ^^
Regression Test Suite: Contract Between Groups
When working on a big project, you often working with a lot of working groups. It will be troublesome if you didn’t manage communication beetwen them well. For instance, if your are at verification group, you will receive RTL codes from designer group, you will need to verify it, and you will need to deliver the codes ; which has been PASSED from verification process before; to implementation group.
Tutorial SystemC+SCV Hello World!
Prerequisite:
1. Familiar with Modelsim. For you who is not familiar yet, please try my tutorial regarding Modelsim here.
2. Familiar with C/C++ and Verilog programming.
3. Not necessarily having experienced in both systemC and assertion technique, because both will be learned here.
Goals:
1. To run a simple program to begin exploration and experiment.
2. To test your platform/PC, whether it already support systemC and SCV or not. In this tutorial, we use Modelsim to run HDL+SysytemC co-simulation, as Modelsim already support systemC and SCV, and GCC as well. For Installation issue, plesea refer to http://www.model.com
Steps:
1. Create working directory and change directory into it.
$ mkdir training
$ cd training
2. open editor program (emacs, kate, gedit, kwrite, or another) then write this code and save it into hello.cpp
#include “systemc.h”
SC_MODULE(hello) {
void prog()
{
cout << “Hello World!” << endl;
}
SC_CTOR(hello){
SC_METHOD(prog);
}
};
SC_MODULE_EXPORT(hello);
3. Create Modelsim working library.
$ vlib work
4. Compile code.
$ sccom -g hello.cpp
5. Link code.
$ sccom -link -B/usr/bin
6. Rum simulation.
$ vsim -do run -c hello
You will see “Hello World!” in your console/screen. It means that your platform/PC already support SystemC, and you’ve already run a simple program of SystemC.
Next you will learn how to implement a simple SCV function in systemC. SCV or SystemC Verification is a collection of code/library to support verification purpose. There are a bunch of randomization library, assertion, ect.
7. Add SCV function to hello.cpp and save it.
#include “systemc.h”
#include “scv/scv.h”
SC_MODULE(hello) {
scv_smart_ptr<int> rand;
void prog()
{
cout << “Hello World!” << endl;
cout << “These are random value from SCV” << endl;
for (int i=0; i<10; i++) {
rand->keep_only(0,10);
rand->next();
cout<<*rand;
}
}
SC_CTOR(hello){
SC_METHOD(prog);
}
};
SC_MODULE_EXPORT(hello);
8. Re-run steps 4-to-6, but this time add “-scv” option within command line.
$ sccom -g -scv hello.cpp
$ sccom -link -scv -B/usr/bin
$ vsim -do run -c hello
You will see ten random value under “Hello World!”. These value are generated from SCV library. Well, till now you’ve been finishing my turorial regarding systemC+SCV Hello World!
Experiment:
prog() is actually a general C function. It represent a subset of SystemC program. Hence, you can modify this function as you want, or you can put/replece another C code there. Simulation will run naturally like you’re runing your C code without SystemC. Enjoy!
What Designers Thought About Assertion
It is obvious that designer has difficulties in migrating their design methodology from conventional to assertion-based design. Assertion often be associated with verification phase, not design phase. “Therefore, adding assertion in design phase often wasting time”, they said.
I once realized this fact when working as a head of verification project. I was trying to migrate our verification approach from test vector auto-compare method to assertion-based method. Assertion was new methodology and wasn’t familiar to us. But, as we learnt, practised, and implemented it in our verification infrastructure, we began to familiar with it, and began to enjoy using it.
However, it went difficult for design engineers. The had resistence. The refuse using assertion in their design phase. Why? These are good examples of most of designer’s thougth about assertion methodologies and identifies the fallacies in reasoning. I take it from a book of “Assertion-Based Design 2nd Ed”, Harry Foster.
Where am I going to find time to write assertions? I don’t even have time to write comments in my code!
Based on our studies and interviews with multiple projects and engineers, the overhead in writing assertions can amount to anywhere between one and three percent extra time added to the RTL coding phase of a design project (note that the RTL coding phase is only one of many phases within a project). In general, the overhead is very minor (that is, closer to one percent). Why? Because, regardless whether you write comments or add assertions prior to or during RTL coding, you are already analyzing and thinking about correct or expected behavior of your design. In other words, you are already considering:
(i) What is the valid interface behavior?
(ii) What are the legal states for my FSM?
(iii) What are the boundary conditions on my queue controller?
By adding assertions, you are formalizing your thought process in a form that is verifiable. In fact, many design errors are avoided prior to RTL coding and verification through this more systematic approach to design and analysis.
I have to get my design working first. If there is time later, then I ’ll add assertions.
This argument translates to: “Assertions will not save me (us) any time”. Yet, experience demonstrates that assertions can save up to 50 percent of debug time [Abarbanel et al. 2000]. These people will change their position when they find that the assertions other engineers used in their blocks effectively isolate failures. When they save debug time just by isolating failures that originate outside their block, they will be convinced of the value of including assertions in their own blocks.
I will spend more time debugging the assertions than debugging my code; therefore, they are a waste of time.
Compare this sentiment to developing testbenches. Does this mean that creating testbenches to verify the design is useless—since the testbenches might also contain bugs? Certainly not, and the same is true with assertions (that is, assertions could contain bugs, but that does not render the practice useless). As with most new experiences, using assertions involves a learning curve. As engineers gain experience with assertions, they recognize what constitutes a correct assertion and spend less time debugging assertions. If engineers are spending time debugging assertions, it is because they did not completely express the assertion, or they did not fully understand some subtle aspect of the design. In fact, the analysis process that takes place while specifying assertions quite often uncovers complex bugs, and this occurs prior to any form of verification.
I can’t think of any assertions to put in my code. There are no places for them.
Designers who say they can’t see any potential errors in their code will be spending a lot of time rewriting their design code. Experienced designers recognize that typographical, transcription, and design errors all contribute to functional problems that must be addressed. Inexperienced designers must be taught how to see the potential for problems. Then, they will see that they can use assertions to naturally check for correctness.
The assertions slow down simulations. They waste time.
During the post-mortem review for a Convex Computer Corporation project2, a survey was given to a design team with the following question: “If we had computers that could simulate the design 10x faster than today, would that help you debug faster?” One astute designer responded, “No! We run all the tests overnight and I arrive in the morning with a list of failures I can’t work through in one day.” This illustrates that the debugging process is where efficiency improvements are required. So, although assertions do have an overhead, that overhead is comparable to monitoring the design for
correctness.
The Cyrix M3 project and the HP ASIC project found that assertions produced an overhead in the range of 15-30 percent [Krolnik 1998] [Foster and Coelho 2001], depending on the number of assertions and amount of monitoring. For example, Cisco reported a 15 percent increase in simulation runtime on a large ASIC project containing 10,656 OVL assertions [Smith 2002]. It is difficult to debug with assertions. It’s not possible to fail an assertion and continue debugging. [Borland 2002] This is actually a critique of the assertion methodology. A good assertion methodology has controls that allow simulations to define an error threshold and controls to determine what action (continue simulation, terminate the simulation, stop, debug) must occur when that threshold is reached. (See Chapter 2, “Assertion Methodology” on page 21 for details on an effective assertion-based verification methodology.)
Designers shouldn’t check their own code. Hence, adding assertions violates this rule.
Adding assertions in the RTL design is a way of specifying expected behavior and it is analogous to the following example of asserting that a pointer will never be null in software designs. Software engineers have known for years that it is a good idea to check that a pointer passed into their code is not null prior to use. If a null pointer is encountered during normal execution, the problem can be quickly isolated when an assertion is used. Notice how asserting that a pointer will not be null says nothing about who is going to test what. In other words, the designer is placing a proposition into the code that states that a particular implementation property will always be TRUE. This is not violating Bergeron’s [2000] redundancy verification convergence model (that is, design engineers should not verify their own designs). In fact, the verification engineer will read the design specification and create a set of test scenarios3 to validate the design.
During the course of verification, if a sequence of events emerges in which a calling routine passes a null pointer, then the problem is quickly isolated via the implementer’s assertion, and this dramatically simplifies debugging. In this respect, RTL design should be no different than software design. When the designer asserts that two signals must always be mutually exclusive, this does not state how the design should be verified. In fact, rarely will the verification engineer focus on implementation-specific testing. Hence, assertions added to the RTL implementation improve the overall quality of the verification process.
Modelsim Basic Tutorial
This post is about tutorial of using Modelsim in Digital VLSI design. I’ve been using this tutorial as an assistant lecturer material of VLSI Design course (EL-4015) at Bandung Institut of Technology in past 2007. Although It has been a long time ago, I think this tutorial is still helpful for you who are beginner in Modelsim. It will give you basic information in an easy way of how you should deal with Modelsim to compile and simulate your design.
This tutorial is using paralel-to-serial as a RTL design example. The RTL is written in Verilog. You will learn about how to compile, simulate, and observe the result in a waveform, both using GUI and console/script. Enjoy!
Download Tutorial.