Movatterモバイル変換


[0]ホーム

URL:


Wayback Machine
55 captures
23 Sep 2008 - 28 Aug 2025
MayFEBJun
Previous capture07Next capture
201720192020
success
fail
COLLECTED BY
Organization:Internet Archive
These crawls are part of an effort to archive pages as they are created and archive the pages that they refer to. That way, as the pages that are referenced are changed or taken from the web, a link to the version that was live when the page was written will be preserved.

Then the Internet Archive hopes that references to these archived pages will be put in place of a link that would be otherwise be broken, or a companion link to allow people to see what was originally intended by a page's authors.

The goal is tofix all broken links on the web. Crawls of supported "No More 404" sites.
This is a collection of web page captures from links added to, or changed on, Wikipedia pages. The idea is to bring a reliability to Wikipedia outlinks so that if the pages referenced by Wikipedia articles are changed, or go away, a reader can permanently find what was originally referred to.

This is part of the Internet Archive's attempt torid the web of broken links.
TIMESTAMPS
loading
The Wayback Machine - https://web.archive.org/web/20190207185018/https://www.codeproject.com/KB/cpp/cpp_generators.aspx
Click here to Skip to main content
13,851,911 members
Sign in
Email
Password

Sign in with
Home
Click here to Skip to main content
Search within:



 
Add your own
alternative version

Stats

67.9K views
25 bookmarked
Posted 20 Sep 2008
LicencedBSD

Generators in C++

, 17 Aug 2015
  3.90 (20 votes)
1 vote, 5.3%
1
1 vote, 5.3%
2
4 votes, 21.1%
3
4 votes, 21.1%
4
10 votes, 52.6%
5
3.90/5 - 20 votes
1 removed
μ 3.81, σa 2.07 [?]
loading...
Rate this:
PleaseSign up or sign in to vote.
The way to add generators with yield to C++

Introduction

As we know, iterators in C++ is a good but not a perfect abstraction. The concept offoreach() (D, Python, Ruby, etc.) appears as a more generic solution. At least,foreach() does not require an artificialiterator::end() to be defined for the collection.

Theforeach() abstraction can be imagined as some function/object that returns the next value of collection/sequence each time it gets invoked. Such functions are known as generators.

The proposed implementation of the generator/yield feature is provided below in full.

Background

This version ofgenerator() for C++ is based on the bright idea ofSimon Tatham - "coroutines in C". In particular, on the idea of usingswitch/case for this implementation.

Declaring a Generator

To declare a generator, you will use$generator,$yield,$emit, and$stop "keywords" that are macro definitions in fact.

And here is a typical implementation of a generator that emits numbers from10 to1 in descending order:

include"generator.h"$generator(descent){// place for all variables used in the generatorint i;// our counter// place the constructor of our generator, e.g.// descent(int minv, int maxv) {...}// from $emit to $stop is a body of our generator:       $emit(int)// will emit int values. Start of body of the generator.for (i =10; i >0; --i)         $yield(i);// a.k.a. yield in Python,// returns next number in [1..10], reversed.   $stop;// stop, end of sequence. End of body of the generator.};

Having such a descending generator declared, we will use it as:

int main(int argc,char* argv[]){  descent gen;for(int n; gen(n);)// "get next" generator invocation    printf("next number is %d\n", n);return0;}

Thegen(n) thing is in fact an invocation of thebool operator()(int& v) method defined "under the hood" of our generator object. It returnstrue if the parameterv was set, andfalse if our generator cannot provide more elements - was stopped.

As you may see,for(int n; gen(n);) looks close enough to the constructionfor(var n in gen) used in JavaScript for exactly the same purpose. Expressiveness is the beauty of the approach.

generator.h

And here is the source code of the generator implementation:

#ifndef __generator_h__#define __generator_h__// generator/continuation for C++// author: Andrew Fedoniouk @ terrainformatica.com// idea borrowed from: "coroutines in C" Simon Tatham,//   http://www.chiark.greenend.org.uk/~sgtatham/coroutines.htmlclass _generator{protected:int _line;public:  _generator():_line(0) {}};#define $generator(NAME) struct NAME : public _generator#define $emit(T) bool operator()(T& _rv) { \switch(_line) {case0:;#define $stop  } _line = 0; return false; }#define $yield(V)     \do {\            _line=__LINE__;\            _rv = (V);returntrue;case __LINE__:;\        }while (0)#endif

That is a bit cryptic, but if you would read the original article of Simon Tatham, then you will get an idea of what is going on here.

Limitations of the Approach

One obvious limitation -$yield cannot be placed inside aswitch as$emit() declares aswitch by itself.

History

This approach of making generators in C++ was originally published in two articles in my blog:

License

This article, along with any associated source code and files, is licensed underThe BSD License

Share

About the Author

c-smile
FounderTerra Informatica Software
CanadaCanada
Andrew Fedoniouk.

MS in Physics and Applied Mathematics.
Designing software applications and systems since 1991.

W3C HTML5 Working Group, Invited Expert.

Terra Informatica Software, Inc.
http://terrainformatica.com

You may also be interested in...

Comments and Discussions

 
FirstPrevNext
QuestionNice trickPin
rrrado24-Aug-15 1:49
memberrrrado24-Aug-15 1:49 
Thanks for sharing, but it is not possible to compile it - the _line declaration must be public or protected.
What is purpose of the
do ....while (0)
? It works also without that.
AnswerRe: Nice trickPin
c-smile24-Aug-15 11:40
memberc-smile24-Aug-15 11:40 
Thanks,protected: section is needed there of course.

As ofdo .... while (0) idiom check this:
http://stackoverflow.com/questions/257418/do-while-0-what-is-it-good-for[^]
SuggestionIdea for a C++11 generatorPin
jaybus5618-Aug-15 5:28
memberjaybus5618-Aug-15 5:28 
I think I found a real C++11 solution (or should I call it "abuse"Wink | ;-) ) without macros that seems to work (at least) with GCC.

// Headers necessary for this example#include<iostream>#include<string>// My generator base class#include<type_traits>#include<iterator>template <typename T>class Generator{public:class Result:public std::iterator<std::forward_iterator_tag,T> {public:    Result( Generator* g): m_g(g) {}    T&operator *() {return m_g->m_t;}    Result& operator++() { m_g->Proceed();return *this;}static Result& End() {static Result r(0);return r;}bool operator!= (const Result& rhs) {return m_g->NotDone();}private:    Generator* m_g;  };friendclass Result;  Generator(const T& t) : m_t(t), m_r(this) {}virtual Result& begin() {return m_r;}  Result& end()   {return m_r.End();}protected:virtualvoid Proceed() =0;virtualbool NotDone() =0;typename std::remove_const<T>::type m_t;  Result m_r;};// An inheritor generating numerics// (similar to the original article)class MyGenI :public Generator<int>{public:  MyGenI(int i) : Generator<int>(i) {}virtualvoid Proceed() { --m_t;}virtualbool NotDone() {return m_t>-5;}};// An inheritor modifying a stringclass MyGenS :public Generator<std::string>{public:  MyGenS(const std::string& s) : Generator<std::string>(s) {}virtualvoid Proceed() { m_t += m_t[cntr]+1; ++cntr;}virtualbool NotDone() {return cntr <10;}private:int cntr =0;};// An inheritor generating a string from console inputclass MyGenC :public Generator<std::string>{public:  MyGenC(std::string s) : Generator<std::string>(s) {}virtualvoid Proceed()  {    std::cin >> m_c;    m_t += m_c;  }virtualbool NotDone() {return !std::cin.fail();}virtual Result& begin()  {if ( m_t.empty()) {      std::cin >> m_c;      m_t += m_c;    }return m_r;  }private:char m_c;};// The testint main(int argc,char* argv[]){  std::cout <<"numeric\n";auto genI = MyGenI{5};for(auto i: genI)    std::cout <<"next value is " << i << std::endl;  std::cout <<"\nstring\n";auto genS = MyGenS{"a"};for(auto s: genS)    std::cout <<"next value is " << s << std::endl;  std::cout <<"\nconsole\n";auto genC = MyGenC{""};for(auto s: genC)    std::cout <<"next value is " << s << std::endl;return0;}


What do you think of this? Of course I "abuse" the mechanism of range for-loops here and maybe in future versions or other implementations of C++ this won't work anymore. But the usage and definition of the inheritors look so natural to me.

But this is just an idea and maybe somebody finds a much better solution...Smile | :)
Ignorantia non est argumentum.


modified 18-Aug-15 12:40pm.

GeneralRe: Idea for a C++11 generatorPin
c-smile20-Aug-15 15:55
memberc-smile20-Aug-15 15:55 
You have missed the main point of generators.

Body of generator is a code with normal flow where $yield allows you to return the value from the middle of algorithm.

Consider the task of enumerating keys in hash map:

$generator(each_key){const hash_map<string,int>& ht;int n;   hash_map<string,int>::chain_item* pi;    each_key(hash_map<string,int>& ht_):ht(ht_) {}    $emit(string)for( n =0; n < ht.collision_table.size(); ++n )     {        pi = ht.collision_table[n];while(pi) {          $yield(pi->key);          pi = pi->next;        }     }   $stop;// stop, end of sequence. End of body of the generator.};


Try to write the same using your approach. You will need to write full state machine in order to do that.

modified 20-Aug-15 23:04pm.

AnswerRe: Idea for a C++11 generatorPin
jaybus5621-Aug-15 4:31
memberjaybus5621-Aug-15 4:31 
c-smile wrote:
You have missed the main point of generators.


Oh, I think I didnot miss the point.Smile | :)

c-smile wrote:
Body of generator is a code with normal flow where $yield allows you to return the value from the middle of algorithm.


That is what my Proceed member function does by a simple return at an appropriate point...

c-smile wrote:
Try to write the same using your approach.


I assume what you are trying to do is:"for all collisions inhash_map show all (and only) colliding keys" (I could not find ahash_map definition in my VS2010 that compiles your code so I don't know if collision_table corresponds to the buckets inunordered_map or if the former corresponds to buckets with size > 1 only.)

Well, because the consumer cannot see when a new collision starts the usage is somewhat limited - but same is true for my implementation...

And so I did what you asked for:
class each_key :public Generator<std::string>{public:typedef std::unordered_map<std::string,int> tMap;private:typedeftypename tMap::const_local_iterator tChainIt;const tMap&     m_ht;size_t          m_n;  tChainIt        m_pi;  tChainIt        m_pie;void next_collision()  {for ( ;m_n < m_ht.bucket_count(); ++m_n ) {if ( m_ht.bucket_size(m_n) <=1)continue;      m_pi = m_ht.begin(m_n);      m_pie = m_ht.end(m_n++);// because we do not pass the for inc anymorebreak;    }    m_t = NotDone() ? m_pi->first : std::string{};  }public:  each_key( tMap& ht_): m_ht(ht_), m_n(0)  {}virtual Result& begin()override  {    m_n =0;    m_pi=m_pie=m_ht.end(0);    next_collision();return m_r;  }virtualvoid Proceed()override  {if ( ++m_pi != m_pie) {      m_t = m_pi->first;return;// that point corresponds to your yield    }    next_collision();  }virtualbool NotDone()override  {return m_pi != m_pie;}};


The Generator class:
#include<type_traits>#include<iterator>template <typename T>class Generator{public:class Result:public std::iterator<std::forward_iterator_tag,T> {public:    Result( Generator* g): m_g(g) {}    T&operator *() {return m_g->m_t;}    Result& operator++() { m_g->Proceed();return *this;}static Result& End() {static Result r(0);return r;}bool operator!= (const Result& rhs) {return m_g->NotDone();}private:    Generator* m_g;  };friendclass Result;  Generator() : m_r(this) {}  Generator(const T& t) : m_t(t), m_r(this) {}virtual Result& begin() {return m_r;}  Result& end()   {return m_r.End();}protected:virtualvoid Proceed() =0;virtualbool NotDone() =0;typename std::remove_const<T>::type m_t;  Result m_r;};


c-smile wrote:
You will need to write full state machine in order to do that.


I wouldn't call that a "full state machine".Smile | :) But I would call that easy to do and - pretty readable: Simple C++. No macros. No switch-case "tricks". You said it in your OP yourself:
That is a bit cryptic, but if you would read the original article of Simon Tatham, then you will get an idea of what is going on here.


modified 23-Aug-15 6:20am.

QuestionWasn't this a tip?Pin
Nelek16-Aug-15 23:35
protectorNelek16-Aug-15 23:35 
It looks there have been a type change while editing your apportation. Would you mind to revert the type to a tip/trick?

If you need information about how to change your submission to a tip, please read:
Code Project Article FAQ - change to tip[^]

Thanks
M.D.V.Wink | ;)

If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.

AnswerRe: Wasn't this a tip?Pin
c-smile17-Aug-15 14:51
memberc-smile17-Aug-15 14:51 
I am not sure I understand your question, or request. Could you elaborate more?
GeneralRe: Wasn't this a tip?Pin
Nelek18-Aug-15 1:24
protectorNelek18-Aug-15 1:24 
The type of the apportation, It was a Tip/Trick and now is under articles.

I guess there was some problem while the last update. That's why I suggested you to review and change back to Tip/Trick using the info of the link

I can try it as well if you have problems
M.D.V.Wink | ;)

If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.

GeneralRe: Wasn't this a tip?Pin
Nelek18-Aug-15 1:28
protectorNelek18-Aug-15 1:28 
I have just realized that it already is corrected. Forget my previous answer
M.D.V.Wink | ;)

If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.

GeneralMy vote of 5Pin
enobayram12-Nov-12 0:13
memberenobayram12-Nov-12 0:13 
A very clever addition to my C++ arsenal, I've been using this daily for a while now and I'm not even planning to switch to the upcoming boost solution for generators, as these macro based ones are incredibly efficient and clean.
QuestionWhat's the use?Pin
Roland Pibinger21-Sep-08 2:32
memberRoland Pibinger21-Sep-08 2:32 
With your current solution you have to write a new generator for each loop. Even if you created pre-defined generators (e.g. for iterating over an integer range) the advantage of this solution wouldn't be obvious.
AnswerRe: What's the use?Pin
c-smile21-Sep-08 8:47
memberc-smile21-Sep-08 8:47 
I cannot say better than author ofthis message:
Here's a brief case: generators let you factor a complex loop,
dividing the value-producing part from the value-consuming part.
Neither part has to be transformed in a non-obvious way. With
iterators alone, you do this by rewriting the value-producing part as
an Iterator class. This obfuscates the iterator implementation.


Below is a practical example from myTIScript implementation.

On each generator invocation it returns next property of the object. Property is key/value pair.
Object contains either flat table of key/value pairs if there are less than 8 such pairs. Otherwise
properties are organized as a hash table.

$generator(each_property){int    i,cnt;   pvalue props;value  prop;   each_property(VM *c,value obj):props(c)    {       props = CsObjectProperties( obj );    }   $emit2(value,value)// will emit key/value pair of the objif (CsHashTableP(props))      {          cnt = CsHashTableSize(props);for (i =0; i< cnt; ++i)           {              prop = CsHashTableElement(props,i);for (; prop != VM::undefinedValue; prop = CsPropertyNext(prop))              {                $yield2 ( CsPropertyTag(prop),CsPropertyValue(prop) );              }          }      }elsefor (; props != VM::undefinedValue; props = CsPropertyNext(props))          {             $yield2 ( CsPropertyTag(props),CsPropertyValue(props) );          }   $stop;// stop, end of sequence. End of body of the generator.};


Enumeration/iteration of hash table is non-trivial process per se. Here it is a matter of writing loop with collision list walker.
Last Visit: 7-Feb-19 8:50     Last Update: 7-Feb-19 8:50Refresh1

General General   News News   Suggestion Suggestion   Question Question   Bug Bug   Answer Answer   Joke Joke   Praise Praise   Rant Rant   Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink |Advertise |Privacy |Cookies |Terms of Use |Mobile
Web06 |2.8.190205.1 |Last Updated 17 Aug 2015
Article Copyright 2008 by c-smile
Everything elseCopyright ©CodeProject, 1999-2019
Layout:fixed|fluid


[8]ページ先頭

©2009-2025 Movatter.jp