Skip to content

Added parFlatMapN #4243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
danicheg merged 9 commits into from
Oct 7, 2022
Merged

Added parFlatMapN #4243

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a0e0efb
Added flatParMapN
TonioGela Jun 17, 2022
fe96ffa
Added flatParMapN tests for 1-arity
TonioGela Jun 18, 2022
f62d98b
Refactored .map.flatten in to .flatmap
TonioGela Jun 21, 2022
997e803
Modified function name as per discussion
TonioGela Sep 17, 2022
06ac73b
Added ParallelSuite tests for parFlatMapN
TonioGela Sep 20, 2022
5e41658
Changed implementation using arity functions
TonioGela Sep 27, 2022
adbb470
Rewrote test for clarity
TonioGela Sep 28, 2022
c8b6cd0
Autogenerating the test function itself
TonioGela Sep 28, 2022
325b4b4
Generalizing all the parFlatMap tests
TonioGela Oct 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions project/Boilerplate.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,9 @@ object Boilerplate {
- /** @group ParMapArity */
- def parMap$arity[M[_], ${`A..N`}, Z]($fparams)(f: (${`A..N`}) => Z)(implicit p: NonEmptyParallel[M]): M[Z] =
- p.flatMap.map(${nestedExpansion.products}) { case ${nestedExpansion.`(a..n)`} => f(${`a..n`}) }
-
- def flatParMap$arity[M[_], ${`A..N`}, Z]($fparams)(f: (${`A..N`}) => M[Z])(implicit p: NonEmptyParallel[M]): M[Z] =
- p.flatMap.flatten(p.flatMap.map(${nestedExpansion.products}) { case ${nestedExpansion.`(a..n)`} => f(${`a..n`}) })

@armanbilge armanbilge Jun 17, 2022 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not directly about this PR, but I'm confused by the code here (also for the existing method above it). Why is it calling p.flatMap.map instead of p.apply.map? Since the Apply is the parallel instance.

Am I confused or is this a bug? Maybe we need some tests using e.g. EitherNec[String, A]?

@satorg satorg Jun 18, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes agree, good catch... So now I wonder too.

@armanbilge armanbilge Jun 18, 2022 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, no this is completely fine. It's because it's calling vanilla map on a single argument (a tuple), which is then unpacked by the case in the lambda. I was mistakenly reading it as calling a mapN-like method, in which case this would be wrong.

The map must be consistent between the flatMap and the apply by law, and using the apply instance would require a conversion to the parallel datatype and back. So it's an optimization to use the one from the flatMap instead.

@armanbilge armanbilge Jun 21, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually ... this is just flatMap ... right? 😂

Suggested change
- p.flatMap.map(${nestedExpansion.products}) { case ${nestedExpansion.`(a..n)`} => f(${`a..n`}) }
-
- def flatParMap$arity[M[_], ${`A..N`}, Z]($fparams)(f: (${`A..N`}) => M[Z])(implicit p: NonEmptyParallel[M]): M[Z] =
- p.flatMap.flatten(p.flatMap.map(${nestedExpansion.products}) { case ${nestedExpansion.`(a..n)`} => f(${`a..n`}) })
- p.flatMap.map(${nestedExpansion.products}) { case ${nestedExpansion.`(a..n)`} => f(${`a..n`}) }
-
- def flatParMap$arity[M[_], ${`A..N`}, Z]($fparams)(f: (${`A..N`}) => M[Z])(implicit p: NonEmptyParallel[M]): M[Z] =
- p.flatMap.flatMap(${nestedExpansion.products}) { case ${nestedExpansion.`(a..n)`} => f(${`a..n`}) }

@TonioGela TonioGela Jun 21, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is! I run the tests locally just to be sure that I was not missing something syntax related :)

|}
"""
}
Expand Down Expand Up @@ -472,6 +475,12 @@ object Boilerplate {
else
s"def parMapN[Z](f: (${`A..N`}) => Z)(implicit p: NonEmptyParallel[M]): M[Z] = Parallel.parMap$arity($tupleArgs)(f)"

val flatParMap =
if (arity == 1)
s"def flatParMap[Z](f: (${`A..N`}) => M[Z])(implicit p: NonEmptyParallel[M]): M[Z] = p.flatMap.flatMap($tupleArgs)(f)"
else
s"def flatParMapN[Z](f: (${`A..N`}) => M[Z])(implicit p: NonEmptyParallel[M]): M[Z] = p.flatMap.flatten(Parallel.parMap$arity($tupleArgs)(f))"

val parTupled =
if (arity == 1) ""
else
Expand All @@ -490,6 +499,7 @@ object Boilerplate {
-private[syntax] final class Tuple${arity}ParallelOps[M[_], ${`A..N`}](private val $tupleTpe) extends Serializable {
- $parMap
- $parTupled
- $flatParMap
-}
|
"""
Expand Down
4 changes: 4 additions & 0 deletions tests/shared/src/test/scala/cats/tests/SyntaxSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,9 +246,13 @@ object SyntaxSuite {
val fb = mock[M[B]]
val fc = mock[M[C]]
val f = mock[(A, B, C) => Z]
val mf = mock[(A, B, C) => M[Z]]

tfabc.parMapN(f)
(fa, fb, fc).parMapN(f)

tfabc.flatParMapN(mf)
(fa, fb, fc).flatParMapN(mf)
Comment thread
TonioGela marked this conversation as resolved.
Outdated
}

def testParallelBi[M[_], F[_], T[_, _]: Bitraverse, A, B, C, D](implicit P: Parallel.Aux[M, F]): Unit = {
Expand Down